Writing Debuggers in Plain Ruby

👋

Genadi Samokovarov

ã‚˛ãƒŠãƒ‡ã‚Ŗ

 

ã‚˛ãƒŠãƒ‡ã‚Ŗ

 

ã‚˛ãƒŠãƒ‡ã‚Ŗ

Ge

ã‚˛ãƒŠãƒ‡ã‚Ŗ

Gena

ã‚˛ãƒŠãƒ‡ã‚Ŗ

Gena

twitter.com/gsamokovarov
github.com/gsamokovarov

Bulgaria

Bulgaria

Sofia

Sofia

17–18 May in Sofia, Bulgaria

НДК

НДК

НДК

Web Console

A debugger that doesn't stop the world 🌍

Included by default since Rails 4.2.

group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'web-console'
  gem 'spring'
end 
group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'web-console' 👈
  gem 'spring'
end 
group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'web-console' 👌
  gem 'spring'
end 
>

Writing Debuggers in Plain Ruby!

Debugger

A debugger (or debugging tool) is a computer program that is used to test and debug other programs.


https://en.wikipedia.org/wiki/Debugger

Debugging tool 🤔

Kernel#puts 😅

👍

I am a puts debuggerer
>

Binding#pry

Binding#irb

👌

Byebug

А Real Debugger! 😂

Interrupts the debugged program.

Gives you control over its execution.

Let's you step-in(to) method calls.

Let's you step-out of method calls.

Let's you step-over of method calls.

>

🙌

debug.rb

🤚

Debugger in the standard library!

Part of the initial CVS to SVN import during January 1998 🕔
require "debug" 

Written in plain Ruby!

Written in plain Ruby!

>

😀

Available everywhere.

GDB like user-interface.

☚ī¸

GDB like user-interface.

How do you invoke it twice in the same program run?

Last significant commit from 2013.

Uses Kernel#set_trace_func

Kernel#set_trace_func

Invokes a proc on significant VM events.

set_trace_func -> event, file, line, id, binding {
  case event
  when "call"
    # A Ruby method is called.
  end
} 
set_trace_func -> event, file, line, id, binding {
  case event
  when "class"
    # A class or module is opened.
  end
} 
set_trace_func -> event, file, line, id, binding {
  case event
  when "end"
    # A class or module is closed.
  end
} 
set_trace_func -> event, file, line, id, binding {
  case event
  when "line"
    # A new line of code is executed.
  end
} 
set_trace_func -> event, file, line, id, binding {
  case event
  when "raised"
    # An exception is raised.
  end
} 
set_trace_func -> event, file, line, id, binding {
  case event
  when "return"
    # A Ruby method returns.
  end
} 

Soft deprecated in favour of the more modern TracePoint API.

Clearly, it's possible to write debuggers in Ruby...

Why is Byebug written in C?

🤔

Performance

Kernel#set_trace_func is slow.

Invokes the proc for any event, even if you don't need to listen to it.

TracePoint API

Provides Object-Oriented API, similar to the functionality of Kernel#set_trace_func.

Can listen only to specific events.

Can easily stop tracing the program.

Available in Ruby and C.

TracePoint.trace(:line, :call, :return) do |trace|
  case trace.event
  when :call
    current.frames << trace.binding
    current.depth += 1
  when :return
    current.frames.pop
    current.depth -= 1
  when :line
    next if current.depth.positive?

    trace.disable

    context = Context.new(*current.frames[0...-1], trace.binding)
    context.start
  end
end 

Events

line

execute code on a new line

class

start a class or module definition

end

finish a class or module definition

call

call a Ruby method

return

return from a Ruby method

c_call

call a C-language routine

c_return

return from a C-language routine

raise

raise an exception

b_call

event hook at block entry

b_return

event hook at block ending

thread_begin

event hook at thread beginning

thread_end

event hook at thread ending

fiber_switch

event hook at fiber switch

🤔

History

ruby-debug

ruby-debug is a fast implementation of the standard debugger debug.rb.

An old debugger for Ruby 1.8 and, eventually, Ruby 1.9.

Ruby 1.8

github.com/ruby-debug/ruby-debug

Rewritten for Ruby 1.9

github.com/ruby-debug/ruby-debug-base-19

Fork for Ruby 2.x ❓

github.com/ruby-debug/cldwalker/debugger

Fork for Ruby 2.x 👌

github.com/ruby-debug/ko1/debugger2

Rewrite for Ruby 2.x 👌

github.com/denofevil/debase

Unexported Ruby headers

github.com/cldwalker/debugger-ruby_core_source

The TracePoint API solves most of the problems that sprouted the existence of ruby-debug.

Writing debuggers back in the days was way more convoluted. đŸ‘ĩ

Post-Mortem Debugging

Post-mortem debugging is debugging of the program after it has already crashed.

Pry-open the VM stack frames

Kernel#caller

Array[String]

/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/workspace.rb:85:in `eval'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/workspace.rb:85:in `evaluate'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/context.rb:385:in `evaluate'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:493:in `block (2 levels) in eval_input'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:647:in `signal_status'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:490:in `block in eval_input'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/ruby-lex.rb:246:in `block (2 levels) in each_top_level_statement'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/ruby-lex.rb:232:in `loop'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/ruby-lex.rb:232:in `block in each_top_level_statement'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/ruby-lex.rb:231:in `catch'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb/ruby-lex.rb:231:in `each_top_level_statement'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:489:in `eval_input'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:428:in `block in run'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:427:in `catch'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:427:in `run'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/2.6.0/irb.rb:383:in `start'
/Users/genadi/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `top (required)'
/Users/genadi/.rbenv/versions/2.6.2/bin/irb:23:in `load'
/Users/genadi/.rbenv/versions/2.6.2/bin/irb:23:in `main' 

Kernel#caller_locations

Array[Thread::Backtrace::Location]

. Thread::Backtrace::Location#absolute_path
. Thread::Backtrace::Location#label
. Thread::Backtrace::Location#base_label
. Thread::Backtrace::Location#lineno
. Thread::Backtrace::Location#path 

Read-only, presentational APIs.

Debug Inspector API

Let's you open the VM stack frames and build Binding objects for them.

CRuby specific and available only in C.

typedef struct rb_debug_inspector_struct rb_debug_inspector_t;
typedef VALUE (*rb_debug_inspector_func_t)(const rb_debug_inspector_t *, void *);

VALUE rb_debug_inspector_open(rb_debug_inspector_func_t func, void *data);
VALUE rb_debug_inspector_frame_self_get(const rb_debug_inspector_t *dc, long index);
VALUE rb_debug_inspector_frame_class_get(const rb_debug_inspector_t *dc, long index);
VALUE rb_debug_inspector_frame_binding_get(const rb_debug_inspector_t *dc, long index);
VALUE rb_debug_inspector_frame_iseq_get(const rb_debug_inspector_t *dc, long index);
VALUE rb_debug_inspector_backtrace_locations(const rb_debug_inspector_t *dc); 
static VALUE st_mSkiptrace;

static VALUE st_current_bindings(VALUE self)
{
  return rb_debug_inspector_open(current_bindings_callback, NULL);
}

void Init_cruby(void)
{
  st_mSkiptrace = rb_define_module("Skiptrace");

  rb_define_singleton_method(st_mSkiptrace, "current_bindings", st_current_bindings, 0);
} 
static VALUE
current_bindings_callback(const rb_debug_inspector_t *context, void *data)
{
  VALUE locations = rb_debug_inspector_backtrace_locations(context);
  VALUE binding, bindings = rb_ary_new();
  long i, length = RARRAY_LEN(locations);

  for (i = 0; i < length; i++) {
    binding = rb_debug_inspector_frame_binding_get(context, i);

    if (!NIL_P(binding)) {
      rb_ary_push(bindings, binding);
    }
  }

  return bindings;
} 

High-level APIs

TracePoint is a high-level API.

Debug Inspector, although in C, is a high-level API.

🤔

Why not expose them to Ruby?

DebugInspector#open

github.com/banister/debug_inspector
require 'debug_inspector'

RubyVM::DebugInspector.open { |dc|
  locs = dc.backtrace_locations

  # you can get depth of stack frame with `locs.size'
  locs.size.times do |i|
    # binding of i-th caller frame (returns a Binding object or nil)
    p dc.frame_binding(i)

    # iseq of i-th caller frame (returns a RubyVM::InstructionSequence object or nil)
    p dc.frame_iseq(i)

    # class of i-th caller frame
    p dc.frame_class(i)
  end
} 

😀

☚ī¸

>

Binding#caller

github.com/banister/binding_of_caller
def a
  var = 10
  b
  puts var
end

def b
  c
end

def c
  binding.of_caller(2).eval('var = :hello')
end

a()

\# OUTPUT
\# => hello 

😀

☚ī¸

Thread::Backtrace::Location#binding

A suggestion by @tom_enebo from RubyKaigi 2018.

backtrace = caller_locations
if trace = backtrace[3]
  # C frames cannot build binding.
  trace.binding
end 
backtrace = caller_locations
if trace = backtrace[3]
  # Have optimizing VM like JRuby or TruffleRuby?
  trace.binding
end 
backtrace = caller_locations
if trace = backtrace[3]
  # Return nil for for such frames.
  trace.binding
end 
backtrace = caller_locations
if trace = backtrace[3]
  # Produce a warning, if you wish.
  trace.binding
end 
backtrace = caller_locations
if trace = backtrace[3]
  # No need for trace <=> binding mapping!
  trace.binding
end 

😀

The APIs can be implemented by the VM maintainers.

The debuggers can become, more or less, UX work that is portable across VMs.

Pros & Cons

Cons

  • Performance!
  • Dangerous operations?

Pros

  • Portable debugging tools!
  • Adoption!

🤲

😎

boogah

Proof of concept debugger written using the TracePoint API in plain Ruby.

Not production ready, purely for educational purposes.

>
class Binding
  def boogah
    context = Boogah::Context.new(self)
    context.start
  end
end 
class Boogah::Context
  attr_accessor :frames, :depth

  def initialize(*frames, depth: 0)
    @frames,@depth = frames, depth
    @repl = REPL.new(current_binding)
  end

  def start; @repl.start(self); end
  def stop; @repl.stop; end

  private

  def current_binding
    @frames[@depth - 1]
  end
end 
class Boogah::REPL
  def initialize(binding)
    IRB.setup(caller_locations.first.path, argv: [])

    @binding = binding
    @workspace = IRB::WorkSpace.new(@binding)
    @irb = IRB::Irb.new(@workspace)
  end

  def start(context)
    @binding.receiver.singleton_class.prepend(Boogah::Commands.new(context))
    @irb.run(IRB.conf)
  end

  def stop
    @irb.context.exit
  end
end 
class Boogah::Commands < Module
  def initialize(current)
    Dir.each_child current_directory.join("commands") do |cmd|
      require_command { "commands/#{cmd}" }
    end
  end

  def command(name, short: nil, &block)
    define_method(name, &block)
    alias_method short, name if short
  end

  private

  def require_command(&block)
    filename = block.call
    filename += ".rb" unless filename.end_with?(".rb")

    block.binding.eval current_directory.join(filename).read
  end
end 
command :next, short: :n do
  TracePoint.trace(:line, :call, :return) do |trace|
    case trace.event
    when :call
      current.frames << trace.binding
      current.depth += 1
    when :return
      current.frames.pop
      current.depth -= 1
    when :line
      next if current.depth.positive?

      trace.disable

      context = Context.new(*current.frames[0...-1], trace.binding)
      context.start
    end
  end

  current.stop
end 
command :step, short: :s do
  TracePoint.trace(:call, :return, :line) do |trace|
    case trace.event
    when :call
      trace.disable

      context = Context.new(*current.frames, trace.binding)
      context.start
    when :return
      current.frames.pop
      current.depth -= 1
    when :line
      next if current.depth.positive?

      trace.disable

      context = Context.new(*current.frames[0...-1], trace.binding)
      context.start
    end
  end

  current.stop
end 
command :up, short: :u do
  TracePoint.trace do |trace|
    trace.disable

    context = Context.new(*current.frames, depth: current.depth - 1)
    context.start
  end

  current.stop
end 
command :down, short: :d do
  TracePoint.trace do |trace|
    trace.disable

    context = Context.new(*current.frames, depth: current.depth + 1)
    context.start
  end

  current.stop
end 
github.com/gsamokovarov/boogah

gem install boogah

TL;DR

Writing debuggers may seem daunting, but we have great APIs to do so.

With a little bit of effort, we can remove the need of lower-level code altogether.

Debuggers can be mostly UX work.

Debuggers can be portable.

Thank you!

🙇‍♂ī¸