MiniTest vs RSpec

I have used both, and these are my arguments for and against each of them:

MiniTest

A good thing about MiniTest is, that it is relatively simple and that it is just plain ruby, so no big amount of magic. Some of the drawbacks are these:

  • you always have to call super in the setup method of your derived classes, which is easy to forget
  • assert_equals(a,b) is hard to read, also i often forget which of the two arguments is the expected one and which is the actual one
  • assert x fails with the very uninformative Assertion failed message
  • how do you run all tests in a subdirectory? rake test TEST=a/b/c does not work and rake test TEST=a/b/c/* runs only the files in the directory c.

RSpec

I have used rspec for the longest. These are my points:

  • you dont have to call super in you before-blocks. they stack up.
  • you dont have the assert_equals(a,b) problem
  • the expect(a).to eq(b) syntax is not so pretty
  • it is easy to clutter up you tests with many nested contexts and let-definitions
  • it comes with a lot of features:
    • focus/filters: only run specific tests
    • colorized output
    • rspec a/b/c runs all tests in that directory and all subdirectories

What I use

I use RSpec, but i usually dont use the RSpec syntax. I use the wrong gem, which makes your assertions look like this:

assert{ a == b } # when this fails it prints out both a and b
assert{ a.special? } # when this fails, it prints out a

You dont have to remember in what order you have to put the arguments. Also you dont have to remember the name of the method that is used to assert that a regex matches some data. In most cases you just use the assert{ ... } style. The assert method of the wrong gem gets rid of the need to write custom failure messages or to use custom assert-methods in order to get additional information when an assertion fails. Look, these do the same:

assert(a == b)
assert_equals(a, b)

Why not just use the first one? Because the output contains no helpful information when the assert fails: Assertion failed, thats all. Thats (I think) why people use assert_equal in that case (or assert_match, assert_valid, assert_xyz). When using assert{ a == b }, you dont need that, since it will print out the values of a and b in the case of failure.

What I use in a full-blown (rails) app

In a full-blown (rails) app, i use gems like factory_girl, database_cleaner, webmock and TimeCop. The setup for all tests is a little complicated:

Timecop.safe_mode = true

default_strategy      = :transaction
integration_strategy  = :truncation

RSpec.configure do |config|

  Rails.logger.level = 3

  Capybara.javascript_driver = :webkit

  config.mock_with :rspec

  config.treat_symbols_as_metadata_keys_with_true_values = true
  config.filter_run :focus => true
  config.run_all_when_everything_filtered = true

  config.before(:suite) do
    DataMapper::Model.descendants.to_a.each do |model|
      DataMapper.repository(:default).adapter.execute "DROP TABLE IF EXISTS #{model.storage_name} CASCADE"
    end if DataMapper.repository(:default).adapter.options[:adapter] == 'postgres'

    DataMapper.finalize.auto_migrate!
    DatabaseCleaner.strategy = default_strategy
  end

  config.around(:each) do |block|
    WebMock.enable!
    Sidekiq::Testing.fake!

    DatabaseCleaner.strategy = if example.metadata[:js]
      integration_strategy
    else
      default_strategy
    end

    DatabaseCleaner.start
    block.run
    DatabaseCleaner.clean
    Sidekiq::Extensions::DelayedMailer.jobs.clear
    ActionMailer::Base.deliveries.clear
  end

  config.before(:each, search: true) do
    Search.indices['search'].recreate
    Search.connection.refresh
  end

  config.around(:each, redis: true) do |block|
    redis.client.reconnect
    block.run
    redis.flushall
  end
end

Each example uses webmock to make sure no external requests are made. If you dont want that, you can call Webmock.disable! in your example. database_cleaner makes sure that the database is cleared after each example. Also the deliveries array of the mailer is cleared. For integration tests that use elasticsearch or redis, there are two metakeys that can be used in order to flag the tests. The around-blocks just make sure that elasticsearch and redis contain no data. You have to really make sure that no state is left over in between your tests at all, because this can cause order-dependent-test-failures and those are sometimes very hard to discover and fix.

I also use factories instead of fixtures. In my tests i only create records i need and i only specify the data that is important for the test. All other fields of the records are filled out with unimportant default data.

I try to avoid mocks and stubs. Huge amount of mocking and stubbing is a sign that you are doing something wrong in your tests or that your code has some code-smells which should be refactored. Also, when you use stubs/mocks and you change your actual implementation, the tests might still be green, even if your implementation is broken (e.g. interfaces dont match exactly). Those errors are hard to find. Dont get me wrong, i use mocks and stubs, they definitely have their uses. But overusing them is a bad idea.

I also try to avoid shared-examples and nested contexts. A context-block for a method is fine, but deeper nesting is a code-smell IMO. I also dont try very hard to DRY up my test setups. Having multiple before-blocks for test-examples is often hard to grasp. I did that in the beginning, but now i try to minimize the use of let and before/after blocks. I also try to avoid the subject/its methods.

In the beginning i tried to adhere to the one-assertion-per-example idiom, but i also moved away from that, especially in integration tests.