jubatus::core::fv_converter::character_ngram Class Reference

#include <character_ngram.hpp>

Inheritance diagram for jubatus::core::fv_converter::character_ngram:

Collaboration diagram for jubatus::core::fv_converter::character_ngram:

Public Member Functions
	character_ngram (size_t length)
void	split (const std::string &string, std::vector< std::pair< size_t, size_t > > &ret_boundaries) const
Public Member Functions inherited from jubatus::core::fv_converter::word_splitter
void	extract (const std::string &text, std::vector< string_feature_element > &result) const
	word_splitter ()
virtual	~word_splitter ()
Public Member Functions inherited from jubatus::core::fv_converter::string_feature
virtual	~string_feature ()

Private Attributes
const size_t	length_

Detailed Description

Definition at line 29 of file character_ngram.hpp.

Constructor & Destructor Documentation

jubatus::core::fv_converter::character_ngram::character_ngram ( size_t  length )

inlineexplicit

Definition at line 31 of file character_ngram.hpp.

      : length_(length) {
  }

Member Function Documentation

void jubatus::core::fv_converter::character_ngram::split ( const std::string &  string, std::vector< std::pair< size_t, size_t > > &  ret_boundaries  ) const

virtual

Implements jubatus::core::fv_converter::word_splitter.

Definition at line 35 of file character_ngram.cpp.

References length_.

                                                               {
  const size_t len = length_;
  std::vector<size_t> queue(len);
  size_t p = 0;
  size_t n = 0;

  std::vector<std::pair<size_t, size_t> > bounds;
  for (size_t i = 1; i <= string.size(); ++i) {
    if (i == string.size() || is_begin_of_character(string[i])) {
      ++n;
      if (n >= len) {
        size_t b = queue[p];
        bounds.push_back(std::make_pair(b, i - b));
      }
      queue[p] = i;
      ++p;
      if (p == len) {
        p = 0;
      }
    }
  }

  bounds.swap(ret_boundaries);
}

Member Data Documentation

const size_t jubatus::core::fv_converter::character_ngram::length_

private

Definition at line 40 of file character_ngram.hpp.

Referenced by split().

---

The documentation for this class was generated from the following files:

• jubatus/core/fv_converter/character_ngram.hpp
• jubatus/core/fv_converter/character_ngram.cpp