License

AeEasy

Description

AeEasy gem collection allow advance AnswersEngine features possible by including a collection of specialized gems.

Install gem:

gem install 'ae_easy'

Require gem:

require 'ae_easy'

Included gems documentation:

ae_easy-core: http://rubydoc.org/gems/ae_easy-core/frames
ae_easy-config: http://rubydoc.org/gems/ae_easy-config/frames
ae_easy-router: http://rubydoc.org/gems/ae_easy-router/frames
ae_easy-text: http://rubydoc.org/gems/ae_easy-text/frames
ae_easy-login: http://rubydoc.org/gems/ae_easy-login/frames

How to implement

Sample AnswersEngine project

Lets take a simple project without ae_easy:

# ./config.yaml

seeder:
  file: ./seeder/seeder.rb
  disabled: false
parsers:
  - page_type: search
    file: ./parsers/search.rb
    disabled: false
  - page_type: product
    file: ./parsers/product.rb
    disabled: false

# ./seeder/seeder.rb

pages << {
  'url' => 'https://example.com/login.rb?query=food',
  'page_type' => 'search'
}

# ./parsers/search.rb

require 'cgi'

html = Nokogiri.HTML content
html.css('.name').each do |element|
  name = element.text.strip
  pages << {
    'url' => "https://example.com/product/#{CGI::escape name}",
    'page_type' => 'product',
    'vars' => {'name' => name}
  }
end

# ./parsers/product.rb

html = Nokogiri.HTML content
description = html.css('.description').first.text.strip
outputs << {
  '_collection' => 'product',
  'name' => page['vars']['name'],
  'description' => description
}

Adding ae_easy to sample project

One of AeEasy's main feature is to allow users to use classes instead of raw scripts with the whole answersengine gem contexts (seeder, parsers, finishers, etc.) functions and objects integreated directly on our classes.

Converting seeders, parsers and finishers to AeEasy supported classes is quite easy, just wrap your seeders and parsers like this:

class MySeeder
  include AeEasy::Core::Plugin::Seeder
  
  # Create "initialize_hook_*" methods instead of "initialize" method
  #  to prevent overriding the logic behind AeEasy
  def initialize_hook_my_seeder opts = {}
    @my_param = opts[:my_param]
  end

  def seed

    # Your seeder code goes here

  end
end

class MyParser
  include AeEasy::Core::Plugin::Parser
  
  # Create "initialize_hook_*" methods instead of "initialize" method
  #  to prevent overriding the logic behind AeEasy
  def initialize_hook_my_parser opts = {}
    @my_param = opts[:my_param]
  end
  
  def parse

    # Your parser code goes here

  end
end

class MyFinisher
  include AeEasy::Core::Plugin::Finisher
  
  # Create "initialize_hook_*" methods instead of "initialize" method
  #  to prevent overriding the logic behind AeEasy
  def initialize_hook_my_parser opts = {}
    @my_param = opts[:my_param]
  end
  
  def finish

    # Your finisher code goes here

  end
end

You can also add initialize_hook_ methods to extend the default initialize provided by AeEasy plugins.

Now let's try this on our sample project's seeders and parsers:

# ./seeder/seeder.rb

module Seeder
  class Seeder
    include AeEasy::Core::Plugin::Seeder

    def seed
      pages << {
        'url' => 'https://example.com/search?query=food',
        'page_type' => 'search'
      }
    end
  end
end

# ./parsers/search.rb

module Parsers
  class Search
    include AeEasy::Core::Plugin::Parser

    def parse
      html = Nokogiri.HTML content
      html.css('.name').each do |element|
        name = element.text.strip
        pages << {
          'url' => "https://example.com/product/#{CGI::escape name}",
          'page_type' => 'product',
          'vars' => {'name' => name}
        }
      end
    end
  end
end

# ./parsers/product.rb

module Parsers
  class Product
    include AeEasy::Core::Plugin::Parser

    def parse
      html = Nokogiri.HTML content
      description = html.css('.description').first.text.strip
      outputs << {
        '_collection' => 'product',
        'name' => page['vars']['name'],
        'description' => description
      }
    end
  end
end

Next step is to add router capabilities to consume these classes. To do this, let's create the routers and require our seeder and parsers classes, like this:

# ./router/seeder.rb

require 'ae_easy/router'
require './seeder/seeder'

AeEasy::Router::Seeder.new.route context: self

# ./router/parser.rb

require 'cgi'
require 'ae_easy/router'
require './parsers/search'
require './parsers/product'

AeEasy::Router::Parser.new.route context: self

Now lets create our ./ae_easy.yaml config file to link our routers to our new seeder and parsers classes:

# ./ae_easy.yaml

router:
  parser:
    routes:
      - page_type: search
        class: Parsers::Search
      - page_type: product
        class: Parsers::Product

  seeder:
    routes:
      - class: Seeder::Seeder

Finally, we need to modify our ./config.yaml to use our routers:

# ./config.yaml

seeder:
  file: ./router/seeder.rb
  disabled: false

parsers:
  - page_type: search
    file: ./router/parser.rb
    disabled: false
  - page_type: product
    file: ./router/parser.rb
    disabled: false

Hurray! you have successfullly implemented AeEasy on your project.

ae_easy

Development

Runtime

AeEasy

Description

How to implement

Sample AnswersEngine project

Adding ae_easy to sample project