Project

lbfgsb

0.0
The project is in a healthy, maintained state
Lbfgsb.rb is a Ruby binding for L-BFGS-B that is a limited-memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
 Dependencies

Runtime

>= 0.9.1
 Project Readme

Lbfgsb.rb

Build Status Gem Version BSD 3-Clause License Documentation

Lbfgsb.rb is a Ruby binding for L-BFGS-B that is a limited-memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables. L-BFGS-B is written in FORTRAN. Author converted the codes into C-lang and call that with Ruby native exntesion.

Installation

Add this line to your application's Gemfile:

gem 'lbfgsb'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install lbfgsb

Notes: lbfgsb.rb uses 32-bit integer for the integer type in its native code. If you want to use 64-bit integer, give the installation option as below:

$ gem install lbfgsb -- --with-use-int64

In adition, if you want to use an external BLAS library for linear algebra on LBFGSB optimization, give the directory and library in the installation options as follows:

$ gem install lbfgsb -- --with-blas-dir=/opt/local/openblas/lib --with-blas-lib=openblas

Usage

Example 1. Logistic Regression

require 'numo/narray'
require 'lbfgsb'
require 'rumale'


## Training:
# Define objective function and its derivative.
obj_fnc = proc do |w, x, y|
  Numo::NMath.log(1 + Numo::NMath.exp(-y * x.dot(w))).sum
end

d_obj_fnc = proc do |w, x, y|
  (y / (1 + Numo::NMath.exp(-y * x.dot(w))) - y).dot(x)
end

# Load dataset.
x, y = Rumale::Dataset.load_libsvm_file('svmguide3')

# Extend variable for intercept.
xb = Numo::DFloat.hstack([x, Numo::DFloat.ones(x.shape[0]).expand_dims(1)])

# Initialize weight vector.
w0 = xb.mean(0)

# Optimize weight vector.
res = Lbfgsb.minimize(fnc: obj_fnc, jcb: d_obj_fnc, x_init: w0, args: [xb, y])
pp res
w = res[:x]


## Testing:
# Load dataset.
x_test, y_test = Rumale::Dataset.load_libsvm_file('svmguide3.t')
xb_test = Numo::DFloat.hstack([x_test, Numo::DFloat.ones(x_test.shape[0]).expand_dims(1)])

# Predict class label.
probs = 1.0 / (Numo::NMath.exp(-xb_test.dot(w)) + 1.0)
predicted = Numo::Int32.zeros(x_test.shape[0])
predicted[probs >= 0.5] = 1
predicted[probs <  0.5] =-1

# Evaluate result.
evaluator = Rumale::EvaluationMeasure::Accuracy.new
puts("Accuracy: %.1f%%" % (100.0 * evaluator.score(predicted, y_test)))
$ gem install rumale
$ wget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/svmguide3
$ wget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/svmguide3.t
$ ruby example1.rb
{:task=>"CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH",
 :x=>
  Numo::DFloat#shape=[23]
[6.36634, -86.6413, -18.9489, -11.4564, -16.6738, -23.9111, -6.19714, ...],
 :fnc=>512.3210359536802,
 :jcb=>
  Numo::DFloat#shape=[23]
[8.01782e-05, -0.000226254, -0.00144915, 4.26451e-05, 0.000677138, ...],
 :n_iter=>623,
 :n_fev=>716,
 :n_jev=>716,
 :success=>true}
Accuracy: 61.0%

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yoshoku/lbfgsb.

License

The gem is available as open source under the terms of the BSD-3-Clause License. In addition, L-BFGS-B carries the following condition for use:

This software is freely available, but we expect that all publications describing work using this software , or all commercial products using it, quote at least one of the references given below. This software is released under the "New BSD License" (aka "Modified BSD License" or "3-clause license").