Code Challenge: Combinations
Given an array of arrays of possible values, enumerate all combinations that can occur, preserving order. For instance:
Given: [[1,2,3], [4,5,6], [7,8,9]], calculate the same result as the code below, but do so with an arbitrary size array:
combos = []
[1,2,3].each do |v1|
[4,5,6].each do |v2|
[7,8,9].each do |v3|
combos << [v1, v2, v3]
end
end
end
combos
Entries can be written using one or more functions, and may optionally be written as a class extension (i.e. Array).
Points will be given on technique, performance/speed, and difficulty of the program style. Winner receives a gold medal made of pure awesome. Submit a comment with a link to a gist of your code to enter. Comments are moderated, so your entry is safe until a winner is chosen.
Announcing a DNS tool without the bullshit
I’ve created a simple DNS tool that allows you to easily link DNS results to share with others. Using a third-party tool is a great way to show that the DNS settings are not being affected by your local network, but most third-party tools are riddled with advertising and unclear tool names, with some even charging a membership to access some of their tools. Our tool uses (and displays in the result) the command line used to generate the results instead. While not everyone will be familiar with these different commands, users can still experiment to learn and discover more about these valuable tools.
The tool is available under our name, dns.norbauer.com, or at its Heroku hostname, dns.heroku.com, where the tool is hosted.
The site is written using the Sinatra framework and the code is open source.
ERD diagrams from Sequel Pro
If you need a diagram of your MySQL database and you’re on a Mac, generating an ERD diagram is quite easy – and completely free. Sequel Pro can export Graphviz dot files, and then all you need is a few tools to create the diagram.
- Install graphviz from MacPorts via your Terminal:
sudo port install graphviz - Install Sequel Pro, run the app, connect to your MySQL server and open the database you’d like to diagram.
- Go to File > Export > Graphviz Dot file, and save the file somewhere convenient.
- Generate an SVG file of your diagram:
dot -Tsvg your_database.dot > your_database.svg - You can open SVG files with Opera, Safari, Illustrator, etc, but you can generate a PNG file in a number of ways. You can try installing ImageMagick or libsrvg from MacPorts, or use Illustrator or Inkscape to open and convert the file.
- ImageMagick:
convert your_database.svg your_database.png - libsrvg:
cat your_database.svg | rsvg-convert -o your_database.png
- ImageMagick:
The result is a basic table-based ERD, but it’s not bad for a few minutes of your time.
Thanks to the geert README for the introduction to this process.
Now on Heroku!
We’re a big fan of Heroku. We love the hassle-free hosting and quick, painless deploys that they offer, the ease of scaling, the performance of their system, and the price point, too. So, we’re now drinking our own champagne and running norbauer.com there. The migration was quick and fairly painless (except for a DNS mistake I made). We’re using two Heroku addons: Sendgrid for our contact form emails and we moved to Exceptional for error handling. We were already using Amazon S3 for image hosting, so data migration was as easy as downloading our production data and running heroku db:push.
We even upgraded from Rails 2.0.1 to Rails 2.3.5 somewhere in there – a matter of upgrading our plugins and renaming application.rb to application_controller.rb. The one gotcha we overlooked was changing formatted_notebooks_url(:atom) to notebooks_url(:format => :atom), but thanks to Exceptional, we caught that error quickly and have resolved the problem.
Storing IP addresses as integers
Tying long sessions to an IP address is a good way to ensure some security for your users. But inevitably, you’ll want to store that IP. While you could, of course, store it as a string, such as "123.125.126.127" (called a “dotted quad”), some developers might prefer saving some space and storing it as a compact integer. There’s a right way and a wrong way to do this. The wrong way is as follows:
'12.34.56.78'.sub('.', '').to_i
This is a bad idea because two IP addresses can result in the same integer: 12.34.56.78 and 123.4.56.78 are both valid IPs. The correct way is as follows:
'12.34.56.78'.split('.').collect(&:to_i).
pack('C*').unpack('N').first
To understand this seemingly obfuscated train wreck, let’s look at the data in this chain after each method call:
'12.34.56.78'.split('.'). #=> ['12', '34', '56', 78']
collect(&:to_i). #=> [12, 34, 56, 78]
pack('C*'). #=> "\f\"8N"
unpack('N'). #=> [203569230]
first #=> 203569230
Many developers are unfamiliar with pack and unpack. These allow you to create and extract data into and out of binary-packed strings. In the third method call, you’ll see the result is "\f\"8N". This strange-looking string is really just 4 bytes of data, the numbers 12, 34, 56, and 78 in binary form, put into a string. We then unpack that 4-byte string into a 4-byte integer (with network byte order).
This is also the same problem solved by the C library method inet_aton, which is implemented as part of Ruby’s IPAddr class. Thus, there is a much simpler alternative:
require 'ipaddr'
IPAddr.new('12.34.56.78').to_i #=> 203569230
But what fun is that!
PS. Yes, you can have multi-line method chains simply by leaving the dot at the end of the line. Ruby then knows to look for a method call on the next line. Both snippets are valid Ruby! You should, of course, indent appropriately.
Snow Leopard: Upgrading for Rails Developers
Upgrading to Snow Leopard is not as easy as Apple would lead you to believe; at least not if you’re a Rails developer. Here are a few select errors you might have encountered:
uninitialized constant MysqlCompat::MysqlRes dlopen(/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/nokogiri.bundle, 9): no suitable image found
These errors seems to stem because OS X is now fully 64-bit, and unfortunately, all your compiled libraries are 32-bit. Whoops!
So, let’s fix them.
Important: These instructions are tailored for Intel 64-bit machines, as only 64-bit machines should have these issues. Any Intel Core 2 machines including recent Macbooks, Macbook Pros, and iMacs, and all Mac Pros should work fine with these instructions.
Install 64-bit MySQL
To fix the MySQL problem, you need install the x86_64 version of MySQL. This will uninstall your old MySQL version, but will not migrate your databases. In order to migrate your data, first make sure MySQL is not running, then:
$ sudo mv /usr/local/mysql/data /usr/local/mysql/data.default $ sudo mv /usr/local/mysql-oldversion/data /usr/local/mysql/data
Start up MySQL and your databases should be in tact.
Reinstall the MySQL gem
Now that you have the correct version of MySQL, you need to reinstall the latest MySQL gem. Make sure to uninstall your current MySQL gem first:
$ sudo gem uninstall mysql $ sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config
Re-install MacPorts
To fix nokogiri (and other gems with external dependencies), you’ll need to fix your Macports install. Unfortunately, the only way to fix Macports is to completely reinstall it. You need to completely delete (or at least move) your /opt/local directory. Once that is done, download and install the latest version of MacPorts for Snow Leopard.
You’ll probably want to install two things pretty quickly: libxml2 for nokogiri and git-core:
$ sudo port install libxml2 $ sudo port install git-core +bash_completion +doc
Re-install nokogiri (and other gems)
Any gem that has a compiled component will need to be reinstalled. nokogiri is just one of the libraries; others include ruby-debug, ruby-prof, and a lot of others. To fix them, just run this command:
$ sudo gem pristine --all
Symbol vs String performance
A more interesting metric to this discussion is the use of strings versus symbols. Fortunately, these types of discussions can easily be solved by benchmarks:
Results under Ruby 1.8.6:
user system total real less no op
String instanciation 9.050000 0.010000 9.060000 ( 9.057162) 5.921219
Symbol use 5.130000 0.000000 5.130000 ( 5.131844) 1.995901
new String#to_sym 14.550000 0.020000 14.570000 ( 14.567466) 11.431523
const String#to_sym 11.960000 0.010000 11.970000 ( 11.967217) 8.831274
String const lookup 6.350000 0.010000 6.360000 ( 6.358697) 3.222754
Symbol const lookup 6.400000 0.010000 6.410000 ( 6.416395) 3.280452
No op 3.130000 0.010000 3.140000 ( 3.135943) n/a
Results under Ruby 1.9.1:
user system total real less no op
String instanciation 9.170000 0.010000 9.180000 ( 9.174451) 4.736163
Symbol use 4.920000 0.010000 4.930000 ( 4.930031) 0.491743
new String#to_sym 18.080000 0.030000 18.110000 ( 18.087386) 13.649098
const String#to_sym 13.940000 0.030000 13.970000 ( 13.954099) 9.515811
String const lookup 4.920000 0.000000 4.920000 ( 4.927497) 0.489209
Symbol const lookup 4.910000 0.020000 4.930000 ( 4.921151) 0.482863
No op 4.440000 0.000000 4.440000 ( 4.438288) n/a
What do these results mean? Well, first you need to subtract out the “no op” results from all the others, which I’ve added as a column above. We can now see that string instantiation takes about 90 nanoseconds, which means about 11000 string instantiations per millisecond. Are symbols faster? Considerably so. But the real lesson here is that these numbers are so small that no one in there right mind should spend time worrying about them.
So please, use symbols when you should use symbols, and otherwise use strings.
Garbage collection thresholds in Ruby
I’ve put together a somewhat extensive collection of scripts and results from looking at how often objects get garbage collected in Ruby 1.8 and Ruby 1.9. Performance metrics are also provided running the script under various different environments, which I found quite fascinating. irb, for instance, is ridiculously slow. Granted, the vast majority of these scripts times are spent performing very inefficient ObjectSpace calls, so the raw numbers should be taken with a grain of salt. The metrics are only interesting for comparison, and are of questionable use.
This was instigated by a String versus Symbol discussion in #ruby on Freenode. The individual was primarily worried about memory usage, and these scripts confirm that strings will get garbage collected often and quickly.
One thing to note is how Ruby 1.8 changes its thresholds for garbage collection under each environment, which may be based on the amount of objects in global space that it cannot garbage collect. Under Ruby 1.9, the results are bit more consistent, although this may be due to better metrics being available in Ruby 1.9.
Now shipping internationally
By popular demand, we’ve added shipping support for 32 additional countries to RubyRags:
Our full country list is: Argentina, Australia, Austria, Belgium, Brazil, Canada, Denmark, Finland, France, Germany, Hungary, Iceland, India, Ireland, Italy, Japan, Luxembourg, Malaysia, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Russian Federation, Singapore, Slovakia, Spain, Sweden, Switzerland, UK, USA
If your country isn’t on the list, contact us and let us know of your dissent!
Remembering lighttpd, nginx, and the Internet as a pipe
We asked ourselves a year ago if lighttpd 1.5.0 was vaporware. It seems that was nearly true. At that time, nginx, apache’s mod_proxy_balancer, and haproxy were flourishing as Rails proxy solutions. The more recent introduction of Phusion Passenger (mod_rails) and Ruby Enterprise Edition, both excellent, free, and open-source products, has now driven most deployments (including our own) away from proxying altogether.
There is—or was—generally two counter-arguments to Passenger. The first is the stability and performance argument, which is well understood and has been discussed at length. I believed it was well summarized by Engine Yard’s discussion on the topic. But this is not the argument I’m not interested in.
My early argument against mod_rails was complexity: first, by using Apache instead of a smaller, simpler web server, and second, using a platform that I don’t comprehend and can’t debug. This argument relates to an old article I wrote: that Rails applications—and applications in general—should act like a pipe. (The original article, by James Duncan Davidson, is so old its only accessible through archive.org). My presumption was that the same argument should apply to a 3rd-party mod_rails solution: I don’t know anything about the app server, so if I have problems, I’m screwed. How is this better than FastCGI?
It is ironic, then, to note the success of Passenger in light of Rails’ history with FastCGI. The reason, as it turns out, basically comes down to two answers:
- Passenger is easy to set up under Apache, and requires less configuration than under nginx
- Passenger always works, in my experience, and thus debugging problems is non-existent.
- As a bonus, if you’re using something like monit to ensure your app stays up, monitoring apache2 is a lot easier than individual mongrel processes.
So, better products are better, and users (including myself) will flock to them once they realize it.
The Future
I do wonder, though, whether history will continue to repeat itself in this regard. Where will the next improvements be? What will be our theory (or justifications, as it seems to turn out) behind those improvements? Personally, I’ve been eying threaded solutions for a long time, and given that mod_rails is a multi-process solution, perhaps threads will move things forward in the future. However, given that Ruby 1.9 is still limited by the Global Interpreter Lock, we might not see a threaded answer for a while. The closest we’ll come in the short term is JRuby. Could it be possible that JRuby is the future? Only time will tell.
