RubyConf 2014

Day 1

Matz’s Keynote

Guilds in Ruby 2.4 allow true concurrency; objects can only belong to one Guild at a time

Video games in Ruby

Gosu

Library provides single class to inherit from; overload 3 methods:

initialize() - game then enters loop
update() - called each frame to update state
draw() - called each from to draw game

Create private functions to abstract out behaviors, while keeping the 3 necessary game functions.

class Game < Gosu::Window
  def initialize
    # game details here
  end

  def update
    # change state here
    handle_action
  end

  def draw
    # draw frame here
  end

  def handle_action
    # encapsulate complicated logic here
  end
end

Author demonstrated animation strategies using Gosu::load_tiles, collision detection using rectangular boxes, and refactoring code into standalone classes

class Enemy
  def initialize
    # ...
  end

  def update
    # ...
  end

  def draw
    # ...
  end
end

And in the game class:

class Game < Gosu::Window
  # ...

  def update
    @enemies.each(&:draw)
  end
end

Resources

opengameart.org - Images
PikoPixel - Pixel editor
Tiled Map Editor - build maps and export to JSON/XML
libgosu.org

Ruby & Go (gorb - Ruby native extensions for Go)

Go language

Data and behavior are separated into structs and functions, respectively. Ruby combines these in classes.

`gorb`

Tool that takes Go code and generates the CRuby API wrapper code to get it loaded into the VM.

Steps:

Write Go structs and functions
gorb creates CRuby wrapper files (.go)
gorb loads Go to compile them into native extensions (.so)
Require native extensions in Ruby code and use them as Ruby classes

package fib

type Fibonacci struct {
}

func (f *Fibonacci) Fib(n int) int {
  return f.fib(n)
}

func (f *Fibonacci) fib(n int) int {
  if n < 2 {
    return n
  }
  return f.fib(n-1) + f.fib(n-2)
}

func IsPrime(n int) bool {
  return true
}

gorbgen test/fib/fib.go
# => test/fib/fib.so

require 'test/fib/fib.so'
Test::Fibonacci.new.fib(10)
# => 55

Questions

Does the C code in the VM dispatch to Go? If so, are there cases where we’d see penalties like loading the Go environment?

.so files include Go runtime environment

What might some implementations be in business-driven Ruby app development? Computationally intensive things like matrix operations?

Resources

github.com/lsegal/gorb
gobyexample.com

A look at hooks

Hook: a method called implicitly by Ruby, never explicitly

Ex.

class Foo
  def initialize
  # ...
  end
end

Foo.new # never call #initialize directly

BasicObject#method_missing - called when method does not exist
Object#respond_to_missing - called with obj.respond_to?(:meth_name)
Module#const_missing - don’t use?
Module#included - called when module is included
Module#extended - called when module is extended
Module#prepended - called when module is prepended (added to start of method ancestry)
Class#inherited - called when class is subclassed/inherited from a class
Module#method_added, Module#method_removed - not used very often as we don’t dynamically add methods often
BasicObject#singleton_method_added, etc - called when you dynamically add methods to an object
Kernel#at_exit - calls when an exception is raised, possibly during normal exit as well?

Objects are implictly tried to be converted during binary operations. Implement things to #to_str to add a hook when the object is converted.

Coercion tells you how to coerce classes to avoid:

> 'foo' + 1
TypeError: no implicit conversion of Fixnum into String

Keyword Args

Background

Pros:

Behaves like hashes.
Tells the user what the arguments are going to be used for.
Position doesn’t matter when you omit certain ones.
Dependency injection is better

Cons:

Non-keyword arguments must be before keyword arguments

Usage

# Bad
Net::HTTP.new('server.com', 732, 'admin', 'monkey72')

# Better
Net::HTTP.new(server: 'server.com', port: 732, username: 'admin', password: 'monkey72')

# Best
Net::HTTP.new(
  server: 'server.com',
  port: 732,
  username: 'admin',
  password: 'monkey72'
)

Splat operator

You can use the splat operator (**) to explode hashes into keyword arguments. This lets the arguments be dynamic depending on a particular case.

@salary = calculate_salary(
  base: 20_000,
  per_mile: 10,
  **user.overrides
)

Logic Programming

Intro

Logic programming is a paradigm alongside object-oriented, imperative, and functional. Tell the computer how the world works and let it generate the solution.

Generally have 3 features: declarative, relational, and inferential.

Declarative aspect is easy in Ruby with method chaining. You declare what functions are going to be performing the work and expose a single interface. E.g. has_many – don’t care about the implementation, it “just does it”.

Relational aspect focuses on intersections of data. SQL is a great example of a relational tool (envision a Venn diagram)

18 + 42 = ?  # Easy
18 + ? = 42  # Doesn't work

sum_r(18, 24, ?) # match ? with 42
sum_r(18, ?, 42) # match ? with 24

Logic programming in Ruby

russell library creates a Logic::Solver class that allows you to assert certain constraints and then call solve.

It seems like this talk hid all of the magic of actually solving the problem in the library and focused more on the public interface of the library. “Look, you have to write very little code”, but it would be interested to see what is going on under the hood.

He created a Sudoku solver that looked like it was using a generic Logic::Solver class not tailored for Sudoku and, by passing in the right constraints, he could solve the Sudoku puzzle. It would be very impressive if he can use the same class extensibly to solve other problems assuming you set the correct constraints.a

Mechanics

Unification - pattern matching of how two things are related to each other. Example: concat_r([m, ?, t], [?]), [m, a, t, z]) #=> [m, a, t, z]

Substitution List - Transitive property where you can point variables at each other and they keep those references. If q == r and r == 8, then q == 8

You devise a strategy of attempting to unify through the substitution list and backtracking/branching when you violate a constraint.

Resources

The Art of Prolog, Leon Sterling & Ehud Shapiro
Russell by the speaker, Gavin McGimpsey

Computer Science: The Good Parts

Linked Lists

The very minimum amount of information you can have. You can only start with the first element, and each element contains the first data and a reference to the next element. In order to get the 5th element, you need to start at the first and walk the list to the 5th.

Binary Tree

Instead of having 1 connection between elements, you have 2. If you pre-sort the data as you go, you approach log^2(n) to find a sorted element.

Graph

Generic term for connected nodes with as many connections as you want. Implemented in maps, social networks, electrical grids, etc.

Big-O Notation

Describes complexity of an algorithm. Using a sorted list, for example, improves search time from O(n) to O(log(n)).

Nested loops are a code smell of a bad algorithm.

Day 2

Implementing Virtual Machines

Examples of virtual machines: .NET runtime, JVM, C.

Moving between the CPU, stack, heap, and buffers are orders of magnitude faster than network calls or disk writes.

Using a stack abstraction (Array in Ruby), we can push things onto the stack. Given a program:

vm = VM.new(
  :push, 13,
  :push, 28,
  :add,
  :print,
  :exit
)

We can define instructions on how to perform the actions. Pass through a compiler first to reject methods that are dangerous.

def load(program)
  @program = compile(program)
end

def compile(program)
  program.collect do |v|
  case
  when v.is_a?(Method)
    # check if it's blacklisted
    # return method
  when methods.include?(v)
    # check if it's blacklisted
    # return method
  else
    # return methjod
  end
end

At this point she put a bunch of C code on the screen, talked about direct vs. indirect threading and said, “Now this is the part that makes my brain hurt.” I gave up trying to understand it all.

Resources

Programming Languages, An Interpreter-Based Approach - Samuel N. Kamin
Java Virtual Machine - Jon Meyer & Troy Downing
Threaded Interpretive Languages

Ruby 3 Concurrency

Concurrency exists in Ruby threads but threads are not executed in parallel.

Guilds: Proposal to add restriction for sharing mutable objects. Eliminates race conditions.

Multi-threading programming is difficult. Hard to debug/replicate, possible deadlocks, and hard to tune performance.

Data Race vs. Race Condition

def transfer1(amount, account_from, account_to)
  if (account_from.balance < amount) return NOPE
  account_to.balance += amount
  account_from.balance -= amount
  return YEP
end

Problem 1: Data Race: lines 3 and 4 do not happen in a single transaction
Problem 2: Race Condition: lines 2 and 3/4 can happen in the wrong order.

Highlights need to synchronize sharing mutable objects.

Comparison to other languages:

Shell/DRuby: Everything is copied to send; which is slow
Erlang/Elixir/FP: Prohibits mutable objects; big incompatibility
Place/Racket: Share only immutable objects; we want to share all objects
Clojure: allow sharing with special protocol; can’t accept a special protocol

Goals for Ruby 3:

Keep compatibility with Ruby 2
Parallel program
Don’t care about locks
Share objects with a fast copy
Share immutable objects
Special objects allow sharing mutable objects

Guild Proposal

Multiple Guilds
Each Guild has a Thread
Each Thread has at least one Fiber

Threads in different guilds can run in parallel; threads in the same guild cannot run in parallel.

Mutable objects have a membership to a particular guild. Guilds cannot access mutable objects in other guilds.

Mutable objects

Guild::Channel is an intercommunication library. Provides 2 communication methods:

#copy
- Objects now exist in both guilds.
#transfer_membership
- Transferred objects are invalidated from original guild.

  obj = "foo"
  ch.move(obj)
  puts obj # Error

Immutable objects

Small objects can be sent as messages and copied. Large objects can be passed by reference.

Applications

Use guilds for pipelines, computational functions (like computing Fibonacci outside the main context of a program)

Goals

How to implement object membership: copy and transfer_membership
- Global variables become guild variables
- Constant and Method tables introduce mutual exclusions

C Extensions

Background

There are many ways to hook into C extensions from MRI. The API really just exposes the Ruby internals. Often times, the C implementation supposes particular extensions which leaves JRuby, Rubinius, etc. unable to proceed.

Methods calls in C extensions are slower than in Ruby, because Ruby invisibly caches the method. C does not offer that.

C extensions cannot be optimized since they are already compiled.

FFI is simple but most people are already writing C extensions.

Truffle

From the website:

The Truffle runtime of JRuby is an experimental implementation of an interpreter for JRuby using the Truffle AST interpreting framework and the Graal compiler. It’s an alternative to the IR interpreter and bytecode compiler. The goal is to be significantly faster, simpler and to have more functionality than other implementations of Ruby.

Oracle wrote a VM called Graal VM (JVM + Graal compiler) and framework Truffle on top of that.

Truffle compiles AST to a single module, turns that into a graph, apply optimizations, and turns that into machine code.

Author wrote a C interpreter that runs alongside the Ruby interpreter. C code is given to LLVM interpreter and turned into simplified C code, which is interpreted and optimized at the same time as Ruby. Truffle/Graal can take both Ruby and C files at this point and optimize them both the same way.

C extension -------> [ LLVM ] -------> file.ll ------- [ interpreter] \
                                                                       --------> [ optimizer ]
Ruby file   -------------------------> file.rb ------- [ interpreter] /

Downside

Need the source code; can’t use a closed-source C extension
FFI still has widespread support

The Little Server That Could

An engineering skill is knowing when to dive into an abstraction vs. when to accept constraints and just use it.

Servers communicate with outside world via OS and conform to a specific API.

The outside world

Server runs in a process that gives you scope to define variables and set state without affecting other programs.

Open a socket
- Everything on UNIX is a file. Sockets are no exception.
- OS uses a File Descriptor for the socket but Ruby abstracts that so we don’t have to pass it around.
- Choose address format: UNIX (tmp/something.sock) vs. INET (192.0.1.1:22)
- Something
- Stream sockets: bi-directional, uses TCP 3-way handshake (SYN-SYNACK-ACK). TCP protocol sends packets with an order flag; client assembles them in order.
- Datagram sockets: uni-directional, uses UDP that doesn’t wait for acknowledgement of receipt. Client may or may not receive packets.
Listen for requests
Handle requests
Reply to client

require 'socket'

def server
  socket = create_server_socket
  loop_and_listen_for_client_requests(socket)
end

def create_server_socket
  Socket.new(:INET, :STREAM)
  # ???
end

The application

Parser

Parser extracts header, body, URL, etc. Ex. Ragel Parser

Rack

Implements #call which takes 1 arg, env, and returns [status, headers, body]

Improvements

Blocking network calls in the normal response means that multiple clients are blocked. We can use UNIX forks to handle the blocking code and allow us to serve multiple concurrent requests.

Forking contains references to the same socket. If one doesn’t exit, the socket will be unavaible to processes longer. When you fork a process, the only memory copied to the new process is that which is unique to the process; otherwise it takes memory from parent process. This copy-on-write.

Pre-2.0 Ruby marked objects as different in child processes so their memory footprint would swell up to the size of parent processes.

Post-2.0 Ruby does not do this, allowing C.O.W. processes to remain small.

Threading

Threads share process resources (like memory); they use less memory and die when the process dies, so no zombie processes. Faster to communicate as you don’t have to go through a UNIX socket.

Downsides: non-atomic operations are not thread-safe.

Example:

def buy_toy
  if @available_cat_toys > 0 # CPU executes this, then hops over to another thread (see below)
    buy_cat_toy
  else
    dont_buy_toy
  end
end

def buy_toy
  if @available_cat_toys > 0 # These 3 lines
    buy_cat_toy              # are executed on
    @available_cat_toys -= 1 # a separate thread
  else
    dont_buy_toy
  end
end

By the time we get back to line 3, there are no more available toys (race condition)

You, the Developer

Throw and trap signals to control flow of application. You can shovel code into trapping an interrupt signal, for example, to exit gracefully.

Twitter Facebook Google+ LinkedIn

Michael Zemel

Day 1

Matz’s Keynote

Video games in Ruby

Gosu

Resources

Ruby & Go (gorb - Ruby native extensions for Go)

Go language

gorb

Questions

Resources

A look at hooks

Keyword Args

Background

Usage

Splat operator

Logic Programming

Intro

Logic programming in Ruby

Mechanics

Resources

Computer Science: The Good Parts

Linked Lists

Binary Tree

Graph

Big-O Notation

Day 2

Implementing Virtual Machines

Resources

Ruby 3 Concurrency

Data Race vs. Race Condition

Guild Proposal

Mutable objects

Immutable objects

Applications

Goals

C Extensions

Background

Truffle

Downside

The Little Server That Could

The outside world

The application

Parser

Rack

Improvements

Threading

You, the Developer

`gorb`