class Minitest::Reporters::MeanTimeReporter

This reporter creates a report providing the average (mean), minimum and maximum times for a test to run. Running this for all your tests will allow you to:

1) Identify the slowest running tests over time as potential candidates

for improvements or refactoring.

2) Identify (and fix) regressions in test run speed caused by changes to

your tests or algorithms in your code.

3) Provide an abundance of statistics to enjoy.

This is achieved by creating a (configurable) 'previous runs' statistics file which is parsed at the end of each run to provide a new (configurable) report. These statistics can be reset at any time by using a simple rake task:

rake reset_statistics

Attributes

all_suite_times[RW]

Public Class Methods

new(options = {}) click to toggle source

@param options [Hash] @option #previous_runs_filename [String] Contains the times for each test

by description. Defaults to '/tmp/minitest_reporters_previous_run'.

@option #report_filename [String] Contains the parsed results for the

last test run. Defaults to '/tmp/minitest_reporters_report'.

@option #show_count [Fixnum] The number of tests to show in the report

summary at the end of the test run. Default is 15.

@option show_progress [Boolean] If true it prints pass/skip/fail marks.

Default is true.

@option show_all_runs [Boolean] If true it shows all recorded suit results.

Default is true.

@option #sort_column [Symbol] One of :avg (default), :min, :max, :last.

Determines the column by which the report summary is sorted.

@option order [Symbol] One of :desc (default), or :asc. By default the

report summary is listed slowest to fastest (:desc). :asc will order
the report summary as fastest to slowest.

@return [Minitest::Reporters::MeanTimeReporter]

# File lib/minitest/reporters/mean_time_reporter.rb, line 54
def initialize(options = {})
  super

  @all_suite_times = []
end
reset_statistics!() click to toggle source

Reset the statistics file for this reporter. Called via a rake task:

rake reset_statistics

@return [Boolean]

# File lib/minitest/reporters/mean_time_reporter.rb, line 33
def self.reset_statistics!
  new.reset_statistics!
end

Public Instance Methods

after_suite(suite) click to toggle source

Copies the suite times from the {Minitest::Reporters::DefaultReporter#after_suite} method, making them available to this class.

@return [Hash<String => Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 65
def after_suite(suite)
  super

  @all_suite_times = @suite_times
end
on_record(test) click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 90
def on_record(test)
  super if options[:show_progress]
end
on_report() click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 94
def on_report
  super if options[:show_progress]
end
on_start() click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 86
def on_start
  super if options[:show_progress]
end
report() click to toggle source

Runs the {Minitest::Reporters::DefaultReporter#report} method and then enhances it by storing the results to the 'previous_runs_filename' and outputs the parsed results to both the 'report_filename' and the terminal.

# File lib/minitest/reporters/mean_time_reporter.rb, line 76
def report
  super

  create_or_update_previous_runs!

  create_new_report!

  write_to_screen!
end
reset_statistics!() click to toggle source

Resets the 'previous runs' file, essentially removing all previous statistics gathered.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 102
def reset_statistics!
  File.open(previous_runs_filename, 'w+') { |f| f.write('') }
end

Private Instance Methods

asc?() click to toggle source

@return [Boolean] Whether the given :order option is :asc.

# File lib/minitest/reporters/mean_time_reporter.rb, line 346
def asc?
  order == :asc
end
avg_label() click to toggle source

@return [String] A yellow 'Avg:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 301
def avg_label
  ANSI::Code.yellow('Avg:')
end
column_sorted_body() click to toggle source

@return [Array<Hash<Symbol => String>>] All of the results sorted by

the :sort_column option. (Defaults to :avg).
# File lib/minitest/reporters/mean_time_reporter.rb, line 172
def column_sorted_body
  runs = options[:show_all_runs] ? previous_run : current_run
  runs.keys.each_with_object([]) do |description, obj|
    timings = previous_run[description]
    size = Array(timings).size
    sum  = Array(timings).inject { |total, x| total + x }
    obj << {
      avg:  (sum / size).round(9),
      min:  Array(timings).min.round(9),
      max:  Array(timings).max.round(9),
      last: Array(timings).last.round(9),
      desc: description,
    }
  end.sort_by { |k| k[sort_column] }
end
create_new_report!() click to toggle source

Creates a new report file in the 'report_filename'. This file contains a line for each test of the following example format: (this is a single line despite explicit wrapping)

Avg: 0.0555555 Min: 0.0498765 Max: 0.0612345 Last: 0.0499421 Description: The test name

Note however the timings are to 9 decimal places, and padded to 12 characters and each label is coloured, Avg (yellow), Min (green), Max (red), Last (multi), and Description (blue). It looks pretty!

The 'Last' label is special in that it will be colour coded depending on whether the last run was faster (bright green) or slower (bright red) or inconclusive (purple). This helps to identify changes on a per run basis.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 286
def create_new_report!
  File.write(report_filename, report_title + report_body)
end
create_or_update_previous_runs!() click to toggle source

Creates a new 'previous runs' file, or updates the existing one with the latest timings.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 248
def create_or_update_previous_runs!
  if previously_ran?
    current_run.each do |description, elapsed|
      new_times = if previous_run[description.to_s]
                    Array(previous_run[description.to_s]) << elapsed
                  else
                    Array(elapsed)
                  end

      previous_run.store(description.to_s, new_times)
    end

    File.write(previous_runs_filename, previous_run.to_yaml)

  else

    File.write(previous_runs_filename, current_run.to_yaml)

  end
end
current_run() click to toggle source

@return [Hash<String => Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 113
def current_run
  Hash[all_suite_times]
end
defaults() click to toggle source

@return [Hash] Sets default values for the filenames used by this class,

and the number of tests to output to output to the screen after each
run.
# File lib/minitest/reporters/mean_time_reporter.rb, line 120
def defaults
  {
    order:                  :desc,
    show_count:             15,
    show_progress:          true,
    show_all_runs:          true,
    sort_column:            :avg,
    previous_runs_filename: File.join(Dir.tmpdir, 'minitest_reporters_previous_run'),
    report_filename:        File.join(Dir.tmpdir, 'minitest_reporters_report'),
  }
end
des_label() click to toggle source

@return [String] A blue 'Description:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 306
def des_label
  ANSI::Code.blue('Description:')
end
desc?() click to toggle source

@return [Boolean] Whether the given :order option is :desc (default).

# File lib/minitest/reporters/mean_time_reporter.rb, line 351
def desc?
  order == :desc
end
max_label() click to toggle source

@return [String] A red 'Max:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 311
def max_label
  ANSI::Code.red('Max:')
end
min_label() click to toggle source

@return [String] A green 'Min:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 316
def min_label
  ANSI::Code.green('Min:')
end
options() click to toggle source

@return [Hash]

# File lib/minitest/reporters/mean_time_reporter.rb, line 189
def options
  defaults.merge!(@options)
end
order() click to toggle source

@raise [Minitest::Reporters::MeanTimeReporter::InvalidOrder]

When the given :order option is invalid.

@return [Symbol] The :order option, or by default; :desc.

# File lib/minitest/reporters/mean_time_reporter.rb, line 358
def order
  orders = [:desc, :asc]

  if orders.include?(options[:order])
    options[:order]

  else
    fail Minitest::Reporters::MeanTimeReporter::InvalidOrder,
         "`:order` option must be one of #{orders.inspect}."

  end
end
order_sorted_body() click to toggle source

@return [String] All of the column-sorted results sorted by the :order

option. (Defaults to :desc).
# File lib/minitest/reporters/mean_time_reporter.rb, line 160
def order_sorted_body
  if desc?
    column_sorted_body.reverse

  elsif asc?
    column_sorted_body

  end
end
previous_run() click to toggle source

@return [Hash<String => Array<Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 200
def previous_run
  @previous_run ||= YAML.load_file(previous_runs_filename)
end
previous_runs_filename() click to toggle source

@return [String] The path to the file which contains all the durations

for each test run. The previous runs file is in YAML format, using the
test name for the key and an array containing the time taken to run
this test for values.
# File lib/minitest/reporters/mean_time_reporter.rb, line 208
def previous_runs_filename
  options[:previous_runs_filename]
end
previously_ran?() click to toggle source

Returns a boolean indicating whether a previous runs file exists.

@return [Boolean]

# File lib/minitest/reporters/mean_time_reporter.rb, line 215
def previously_ran?
  File.exist?(previous_runs_filename)
end
rate(run, min, max) click to toggle source

@param run [Float] The last run time. @param min [Float] The minimum run time. @param max [Float] The maximum run time. @return [Symbol] One of :faster, :slower or :inconclusive.

# File lib/minitest/reporters/mean_time_reporter.rb, line 335
def rate(run, min, max)
  if run == min
    :faster
  elsif run == max
    :slower
  else
    :inconclusive
  end
end
report_body() click to toggle source

The report itself. Displays statistics about all runs, ideal for use with the Unix 'head' command. Listed in slowest average descending order.

@return [String]

# File lib/minitest/reporters/mean_time_reporter.rb, line 146
def report_body
  order_sorted_body.each_with_object([]) do |result, obj|
    rating = rate(result[:last], result[:min], result[:max])

    obj << "#{avg_label} #{result[:avg].to_s.ljust(12)} " \
           "#{min_label} #{result[:min].to_s.ljust(12)} " \
           "#{max_label} #{result[:max].to_s.ljust(12)} " \
           "#{run_label(rating)} #{result[:last].to_s.ljust(12)} " \
           "#{des_label} #{result[:desc]}\n"
  end.join
end
report_filename() click to toggle source

@return [String] The path to the file which contains the parsed test

results. The results file contains a line for each test with the
average time of the test, the minimum time the test took to run,
the maximum time the test took to run and a description of the test
(which is the test name as emitted by Minitest).
# File lib/minitest/reporters/mean_time_reporter.rb, line 224
def report_filename
  options[:report_filename]
end
report_title() click to toggle source

Added to the top of the report file and to the screen output.

@return [String]

# File lib/minitest/reporters/mean_time_reporter.rb, line 135
def report_title
  "\n\e[4mMinitest Reporters: Mean Time Report\e[24m " \
  "(Samples: #{samples}, Order: #{sort_column.inspect} " \
  "#{order.inspect})\n"
end
run_label(rating) click to toggle source

@param rating [Symbol] One of :faster, :slower or :inconclusive. @return [String] A purple 'Last:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 322
def run_label(rating)
  case rating
  when :faster then ANSI::Code.green('Last:')
  when :slower then ANSI::Code.red('Last:')
  else
    ANSI::Code.magenta('Last:')
  end
end
samples() click to toggle source

A barbaric way to find out how many runs are in the previous runs file; this method takes the first test listed, and counts its samples trusting (naively) all runs to be the same number of samples. This will produce incorrect averages when new tests are added, so it is advised to restart the statistics by removing the 'previous runs' file. A rake task is provided to make this more convenient.

rake reset_statistics

@return [Fixnum]

# File lib/minitest/reporters/mean_time_reporter.rb, line 238
def samples
  return 1 unless previous_run.first[1].is_a?(Array)

  previous_run.first[1].size
end
show_count() click to toggle source

@return [Fixnum] The number of tests to output to output to the screen

after each run.
# File lib/minitest/reporters/mean_time_reporter.rb, line 195
def show_count
  options[:show_count]
end
sort_column() click to toggle source

@raise [Minitest::Reporters::MeanTimeReporter::InvalidSortColumn]

When the given :sort_column option is invalid.

@return [Symbol] The :sort_column option, or by default; :avg.

# File lib/minitest/reporters/mean_time_reporter.rb, line 374
def sort_column
  sort_columns = [:avg, :min, :max, :last]

  if sort_columns.include?(options[:sort_column])
    options[:sort_column]

  else
    fail Minitest::Reporters::MeanTimeReporter::InvalidSortColumn,
         "`:sort_column` option must be one of #{sort_columns.inspect}."

  end
end
write_to_screen!() click to toggle source

Writes a number of tests (configured via the 'show_count' option) to the screen after creating the report. See '#create_new_report!' for example output information.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 295
def write_to_screen!
  puts report_title
  puts report_body.lines.take(show_count)
end