Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compilation issues #512

Closed
jwijffels opened this issue Jun 24, 2020 · 6 comments
Closed

compilation issues #512

jwijffels opened this issue Jun 24, 2020 · 6 comments
Labels
execution environment Any issues related to execution environment, installation help wanted

Comments

@jwijffels
Copy link

jwijffels commented Jun 24, 2020 •
edited

Hello again. I'm trying to upgrade the R package https://CRAN.R-project.org/package=sentencepiece which runs sentencepiece 0.1.84 to the current version so that it has BPE regularisation.
While doing that in Windows using the mingw compiler, I encountered the following compilation issues - looks mainly related to the use of absl strings.
Are you aware of these issues? Any plans on providing fixes?

This is the overview of the issues

Knipsel

This is the full compilation trace

* installing to library 'C:/Users/Jan/Documents/R/win-library/4.0'
* installing *source* package 'sentencepiece' ...
** using staged installation
** libs
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/arena.cc -o third_party/protobuf-lite/arena.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/arenastring.cc -o third_party/protobuf-lite/arenastring.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/bytestream.cc -o third_party/protobuf-lite/bytestream.o
In file included from ./third_party/protobuf-lite/google/protobuf/stubs/common.h:51,
                 from ./third_party/protobuf-lite/google/protobuf/stubs/bytestream.h:56,
                 from third_party/protobuf-lite/bytestream.cc:31:
third_party/protobuf-lite/bytestream.cc: In member function 'virtual void google::protobuf::strings::ArrayByteSource::Skip(size_t)':
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:184:50: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long long unsigned int'} and 'google::protobuf::stringpiece_ssize_type' {aka 'long long int'} [-Wsign-compare]
 #define GOOGLE_DCHECK_LE(A, B) GOOGLE_DCHECK((A) <= (B))
                                              ~~~~^~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:149:5: note: in definition of macro 'GOOGLE_LOG_IF'
   !(CONDITION) ? (void)0 : GOOGLE_LOG(LEVEL)
     ^~~~~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:179:48: note: in expansion of macro 'GOOGLE_CHECK'
 #define GOOGLE_DCHECK(EXPRESSION) while(false) GOOGLE_CHECK(EXPRESSION)
                                                ^~~~~~~~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:184:32: note: in expansion of macro 'GOOGLE_DCHECK'
 #define GOOGLE_DCHECK_LE(A, B) GOOGLE_DCHECK((A) <= (B))
                                ^~~~~~~~~~~~~
third_party/protobuf-lite/bytestream.cc:155:3: note: in expansion of macro 'GOOGLE_DCHECK_LE'
   GOOGLE_DCHECK_LE(n, input_.size());
   ^~~~~~~~~~~~~~~~
third_party/protobuf-lite/bytestream.cc: In member function 'virtual google::protobuf::StringPiece google::protobuf::strings::LimitByteSource::Peek()':
third_party/protobuf-lite/bytestream.cc:175:20: warning: comparison of integer expressions of different signedness: 'google::protobuf::stringpiece_ssize_type' {aka 'long long int'} and 'size_t' {aka 'long long unsigned int'} [-Wsign-compare]
   if (piece.size() > limit_) {
       ~~~~~~~~~~~~~^~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/coded_stream.cc -o third_party/protobuf-lite/coded_stream.o
third_party/protobuf-lite/coded_stream.cc: In member function 'bool google::protobuf::io::CodedInputStream::ReadLittleEndian32Fallback(google::protobuf::uint32*)':
third_party/protobuf-lite/coded_stream.cc:283:20: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
   if (BufferSize() >= sizeof(*value)) {
       ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
third_party/protobuf-lite/coded_stream.cc: In member function 'bool google::protobuf::io::CodedInputStream::ReadLittleEndian64Fallback(google::protobuf::uint64*)':
third_party/protobuf-lite/coded_stream.cc:300:20: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
   if (BufferSize() >= sizeof(*value)) {
       ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
third_party/protobuf-lite/coded_stream.cc: In member function 'void google::protobuf::io::CodedOutputStream::WriteLittleEndian32(google::protobuf::uint32)':
third_party/protobuf-lite/coded_stream.cc:714:32: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
   bool use_fast = buffer_size_ >= sizeof(value);
                   ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
third_party/protobuf-lite/coded_stream.cc: In member function 'void google::protobuf::io::CodedOutputStream::WriteLittleEndian64(google::protobuf::uint64)':
third_party/protobuf-lite/coded_stream.cc:729:32: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
   bool use_fast = buffer_size_ >= sizeof(value);
                   ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/common.cc -o third_party/protobuf-lite/common.o
third_party/protobuf-lite/common.cc: In member function 'google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long int)':
third_party/protobuf-lite/common.cc:246:45: warning: unknown conversion type character 'l' in format [-Wformat=]
 DECLARE_STREAM_OPERATOR(long long         , "%" GOOGLE_LL_FORMAT "d")
                                             ^~~~~~~~~~~~~~~~~~~~~~~~
third_party/protobuf-lite/common.cc:232:38: note: in definition of macro 'DECLARE_STREAM_OPERATOR'
     snprintf(buffer, sizeof(buffer), FORMAT, value);                \
                                      ^~~~~~
third_party/protobuf-lite/common.cc:246:45: warning: too many arguments for format [-Wformat-extra-args]
 DECLARE_STREAM_OPERATOR(long long         , "%" GOOGLE_LL_FORMAT "d")
                                             ^~~~~~~~~~~~~~~~~~~~~~~~
third_party/protobuf-lite/common.cc:232:38: note: in definition of macro 'DECLARE_STREAM_OPERATOR'
     snprintf(buffer, sizeof(buffer), FORMAT, value);                \
                                      ^~~~~~
third_party/protobuf-lite/common.cc: In member function 'google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long unsigned int)':
third_party/protobuf-lite/common.cc:247:45: warning: unknown conversion type character 'l' in format [-Wformat=]
 DECLARE_STREAM_OPERATOR(unsigned long long, "%" GOOGLE_LL_FORMAT "u")
                                             ^~~~~~~~~~~~~~~~~~~~~~~~
third_party/protobuf-lite/common.cc:232:38: note: in definition of macro 'DECLARE_STREAM_OPERATOR'
     snprintf(buffer, sizeof(buffer), FORMAT, value);                \
                                      ^~~~~~
third_party/protobuf-lite/common.cc:247:45: warning: too many arguments for format [-Wformat-extra-args]
 DECLARE_STREAM_OPERATOR(unsigned long long, "%" GOOGLE_LL_FORMAT "u")
                                             ^~~~~~~~~~~~~~~~~~~~~~~~
third_party/protobuf-lite/common.cc:232:38: note: in definition of macro 'DECLARE_STREAM_OPERATOR'
     snprintf(buffer, sizeof(buffer), FORMAT, value);                \
                                      ^~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/extension_set.cc -o third_party/protobuf-lite/extension_set.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/generated_message_table_driven_lite.cc -o third_party/protobuf-lite/generated_message_table_driven_lite.o
In file included from third_party/protobuf-lite/generated_message_table_driven_lite.cc:31:
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h: In instantiation of 'void google::protobuf::internal::ResetOneofField(const google::protobuf::internal::ParseTable&, int, google::protobuf::Arena*, google::protobuf::MessageLite*, google::protobuf::uint32*, google::protobuf::int64, const void*) [with google::protobuf::internal::ProcessingType field_type = (google::protobuf::internal::ProcessingType)0; google::protobuf::uint32 = unsigned int; google::protobuf::int64 = long long int]':
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h:520:49:   required from 'bool google::protobuf::internal::MergePartialFromCodedStreamImpl(google::protobuf::MessageLite*, const google::protobuf::internal::ParseTable&, google::protobuf::io::CodedInputStream*) [with UnknownFieldHandler = google::protobuf::internal::{anonymous}::UnknownFieldHandlerLite; InternalMetadata = google::protobuf::internal::InternalMetadataWithArenaLite]'
third_party/protobuf-lite/generated_message_table_driven_lite.cc:104:24:   required from here
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h:195:19: warning: comparison of integer expressions of different signedness: 'google::protobuf::uint32' {aka 'unsigned int'} and 'int' [-Wsign-compare]
   if (*oneof_case == field_number) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h: In instantiation of 'void google::protobuf::internal::ResetOneofField(const google::protobuf::internal::ParseTable&, int, google::protobuf::Arena*, google::protobuf::MessageLite*, google::protobuf::uint32*, google::protobuf::int64, const void*) [with google::protobuf::internal::ProcessingType field_type = (google::protobuf::internal::ProcessingType)4; google::protobuf::uint32 = unsigned int; google::protobuf::int64 = long long int]':
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h:719:50:   required from 'bool google::protobuf::internal::MergePartialFromCodedStreamImpl(google::protobuf::MessageLite*, const google::protobuf::internal::ParseTable&, google::protobuf::io::CodedInputStream*) [with UnknownFieldHandler = google::protobuf::internal::{anonymous}::UnknownFieldHandlerLite; InternalMetadata = google::protobuf::internal::InternalMetadataWithArenaLite]'
third_party/protobuf-lite/generated_message_table_driven_lite.cc:104:24:   required from here
./third_party/protobuf-lite/google/protobuf/generated_message_table_driven_lite.h:195:19: warning: comparison of integer expressions of different signedness: 'google::protobuf::uint32' {aka 'unsigned int'} and 'int' [-Wsign-compare]
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/generated_message_util.cc -o third_party/protobuf-lite/generated_message_util.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/implicit_weak_message.cc -o third_party/protobuf-lite/implicit_weak_message.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/int128.cc -o third_party/protobuf-lite/int128.o
third_party/protobuf-lite/int128.cc: In function 'std::ostream& google::protobuf::operator<<(std::ostream&, const google::protobuf::uint128&)':
third_party/protobuf-lite/int128.cc:187:13: warning: comparison of integer expressions of different signedness: 'std::streamsize' {aka 'long long int'} and 'std::__cxx11::basic_string<char>::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
   if (width > rep.size()) {
       ~~~~~~^~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/io_win32.cc -o third_party/protobuf-lite/io_win32.o
third_party/protobuf-lite/io_win32.cc: In function 'std::__cxx11::wstring google::protobuf::internal::win32::{anonymous}::normalize(std::__cxx11::wstring)':
third_party/protobuf-lite/io_win32.cc:195:21: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<std::__cxx11::basic_string<wchar_t> >::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
   for (int i = 0; i < segments.size(); ++i) {
                   ~~^~~~~~~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/message_lite.cc -o third_party/protobuf-lite/message_lite.o
third_party/protobuf-lite/message_lite.cc: In member function 'bool google::protobuf::MessageLite::SerializePartialToCodedStream(google::protobuf::io::CodedOutputStream*) const':
third_party/protobuf-lite/message_lite.cc:247:22: warning: comparison of integer expressions of different signedness: 'long long int' and 'const size_t' {aka 'const long long unsigned int'} [-Wsign-compare]
     if (end - buffer != size) {
         ~~~~~~~~~~~~~^~~~~~~
third_party/protobuf-lite/message_lite.cc:259:48: warning: comparison of integer expressions of different signedness: 'int' and 'const size_t' {aka 'const long long unsigned int'} [-Wsign-compare]
     if (final_byte_count - original_byte_count != size) {
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
third_party/protobuf-lite/message_lite.cc: In member function 'bool google::protobuf::MessageLite::AppendPartialToString(std::__cxx11::string*) const':
third_party/protobuf-lite/message_lite.cc:297:19: warning: comparison of integer expressions of different signedness: 'long long int' and 'size_t' {aka 'long long unsigned int'} [-Wsign-compare]
   if (end - start != byte_size) {
       ~~~~~~~~~~~~^~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/repeated_field.cc -o third_party/protobuf-lite/repeated_field.o
In file included from ./third_party/protobuf-lite/google/protobuf/stubs/common.h:51,
                 from ./third_party/protobuf-lite/google/protobuf/stubs/casts.h:36,
                 from ./third_party/protobuf-lite/google/protobuf/repeated_field.h:57,
                 from third_party/protobuf-lite/repeated_field.cc:37:
third_party/protobuf-lite/repeated_field.cc: In member function 'void** google::protobuf::internal::RepeatedPtrFieldBase::InternalExtend(int)':
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:157:48: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
 #define GOOGLE_CHECK_LE(A, B) GOOGLE_CHECK((A) <= (B))
                                            ~~~~^~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:149:5: note: in definition of macro 'GOOGLE_LOG_IF'
   !(CONDITION) ? (void)0 : GOOGLE_LOG(LEVEL)
     ^~~~~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:157:31: note: in expansion of macro 'GOOGLE_CHECK'
 #define GOOGLE_CHECK_LE(A, B) GOOGLE_CHECK((A) <= (B))
                               ^~~~~~~~~~~~
third_party/protobuf-lite/repeated_field.cc:57:3: note: in expansion of macro 'GOOGLE_CHECK_LE'
   GOOGLE_CHECK_LE(new_size,
   ^~~~~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/status.cc -o third_party/protobuf-lite/status.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/statusor.cc -o third_party/protobuf-lite/statusor.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/stringpiece.cc -o third_party/protobuf-lite/stringpiece.o
third_party/protobuf-lite/stringpiece.cc: In member function 'bool google::protobuf::StringPiece::contains(google::protobuf::StringPiece) const':
third_party/protobuf-lite/stringpiece.cc:99:21: warning: comparison of integer expressions of different signedness: 'google::protobuf::stringpiece_ssize_type' {aka 'long long int'} and 'const size_type' {aka 'const long long unsigned int'} [-Wsign-compare]
   return find(s, 0) != npos;
          ~~~~~~~~~~~^~~~~~~
third_party/protobuf-lite/stringpiece.cc: In member function 'google::protobuf::StringPiece google::protobuf::StringPiece::substr(google::protobuf::StringPiece::size_type, google::protobuf::StringPiece::size_type) const':
third_party/protobuf-lite/stringpiece.cc:260:11: warning: comparison of integer expressions of different signedness: 'google::protobuf::StringPiece::size_type' {aka 'long long unsigned int'} and 'const stringpiece_ssize_type' {aka 'const long long int'} [-Wsign-compare]
   if (pos > length_) pos = length_;
       ~~~~^~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/stringprintf.cc -o third_party/protobuf-lite/stringprintf.o
third_party/protobuf-lite/stringprintf.cc: In function 'std::__cxx11::string google::protobuf::StringPrintfVector(const char*, const std::vector<std::__cxx11::basic_string<char> >&)':
third_party/protobuf-lite/stringprintf.cc:151:21: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<std::__cxx11::basic_string<char> >::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
   for (int i = 0; i < v.size(); ++i) {
                   ~~^~~~~~~~~~
third_party/protobuf-lite/stringprintf.cc:154:28: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int' [-Wsign-compare]
   for (int i = v.size(); i < GOOGLE_ARRAYSIZE(cstr); ++i) {
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/structurally_valid.cc -o third_party/protobuf-lite/structurally_valid.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/strutil.cc -o third_party/protobuf-lite/strutil.o
third_party/protobuf-lite/strutil.cc: In function 'void google::protobuf::CEscapeAndAppend(google::protobuf::StringPiece, std::__cxx11::string*)':
third_party/protobuf-lite/strutil.cc:581:19: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long long unsigned int'} and 'google::protobuf::stringpiece_ssize_type' {aka 'long long int'} [-Wsign-compare]
   if (escaped_len == src.size()) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~
third_party/protobuf-lite/strutil.cc: In function 'int google::protobuf::memcasecmp(const char*, const char*, size_t)':
third_party/protobuf-lite/strutil.cc:1306:22: warning: comparison of integer expressions of different signedness: 'int' and 'size_t' {aka 'long long unsigned int'} [-Wsign-compare]
   for ( int i = 0; i < len; i++ ) {
                    ~~^~~~~
third_party/protobuf-lite/strutil.cc: In function 'int google::protobuf::GlobalReplaceSubstring(const string&, const string&, std::__cxx11::string*)':
third_party/protobuf-lite/strutil.cc:1629:18: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<char>::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
        match_pos != string::npos;
        ~~~~~~~~~~^~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/time.cc -o third_party/protobuf-lite/time.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/wire_format_lite.cc -o third_party/protobuf-lite/wire_format_lite.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/zero_copy_stream.cc -o third_party/protobuf-lite/zero_copy_stream.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/protobuf-lite/zero_copy_stream_impl_lite.cc -o third_party/protobuf-lite/zero_copy_stream_impl_lite.o
third_party/protobuf-lite/zero_copy_stream_impl_lite.cc: In member function 'virtual bool google::protobuf::io::StringOutputStream::Next(void**, int*)':
third_party/protobuf-lite/zero_copy_stream_impl_lite.cc:155:16: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<char>::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
   if (old_size < target_->capacity()) {
       ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
In file included from ./third_party/protobuf-lite/google/protobuf/stubs/common.h:51,
                 from ./third_party/protobuf-lite/google/protobuf/io/zero_copy_stream.h:111,
                 from ./third_party/protobuf-lite/google/protobuf/io/zero_copy_stream_impl_lite.h:50,
                 from third_party/protobuf-lite/zero_copy_stream_impl_lite.cc:35:
third_party/protobuf-lite/zero_copy_stream_impl_lite.cc: In member function 'virtual void google::protobuf::io::StringOutputStream::BackUp(int)':
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:157:48: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<char>::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
 #define GOOGLE_CHECK_LE(A, B) GOOGLE_CHECK((A) <= (B))
                                            ~~~~^~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:149:5: note: in definition of macro 'GOOGLE_LOG_IF'
   !(CONDITION) ? (void)0 : GOOGLE_LOG(LEVEL)
     ^~~~~~~~~
./third_party/protobuf-lite/google/protobuf/stubs/logging.h:157:31: note: in expansion of macro 'GOOGLE_CHECK'
 #define GOOGLE_CHECK_LE(A, B) GOOGLE_CHECK((A) <= (B))
                               ^~~~~~~~~~~~
third_party/protobuf-lite/zero_copy_stream_impl_lite.cc:184:3: note: in expansion of macro 'GOOGLE_CHECK_LE'
   GOOGLE_CHECK_LE(count, target_->size());
   ^~~~~~~~~~~~~~~
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c third_party/absl/strings/string_view.cc -o third_party/absl/strings/string_view.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c sentencepiece/src/builtin_pb/sentencepiece.pb.cc -o sentencepiece/src/builtin_pb/sentencepiece.pb.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c sentencepiece/src/builtin_pb/sentencepiece_model.pb.cc -o sentencepiece/src/builtin_pb/sentencepiece_model.pb.o
"C:/rtools40/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c sentencepiece/src/bpe_model.cc -o sentencepiece/src/bpe_model.o
In file included from sentencepiece/src/normalizer.h:27,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
./third_party/absl/strings/string_view.h:152:7: error: redefinition of 'class absl::string_view'
 class string_view {
       ^~~~~~~~~~~
In file included from sentencepiece/src/normalizer.h:26,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/sentencepiece_processor.h:32:7: note: previous definition of 'class absl::string_view'
 class string_view {
       ^~~~~~~~~~~
In file included from sentencepiece/src/normalizer.h:27,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
./third_party/absl/strings/string_view.h:549:58: error: 'npos' is not a member of 'absl::string_view'
                                  size_t n = string_view::npos) {
                                                          ^~~~
./third_party/absl/strings/string_view.h: In function 'absl::string_view absl::ClippedSubstr(absl::string_view, size_t, size_t)':
./third_party/absl/strings/string_view.h:551:12: error: 'class absl::string_view' has no member named 'substr'
   return s.substr(pos, n);
            ^~~~~~
In file included from sentencepiece/src/model_interface.h:31,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h: In member function 'size_t sentencepiece::string_util::string_view_hash::operator()(const absl::string_view&) const':
sentencepiece/src/util.h:57:39: error: no match for 'operator[]' (operand types are 'const absl::string_view' and 'size_t' {aka 'long long unsigned int'})
       hash = ((hash << 5) + hash) + sp[i];
                                       ^
sentencepiece/src/util.h: In function 'bool sentencepiece::string_util::lexical_cast(absl::string_view, Target*) [with Target = bool]':
sentencepiece/src/util.h:73:44: error: no matching function for call to 'std::__cxx11::basic_string<char>::basic_string(absl::string_view&)'
   std::string lower_value = std::string(arg);
                                            ^
In file included from C:/rtools40/mingw64/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw64/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:614:9: note: candidate: 'template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&)'
         basic_string(_InputIterator __beg, _InputIterator __end,
         ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:614:9: note:   template argument deduction/substitution failed:
In file included from sentencepiece/src/model_interface.h:31,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h:73:44: note:   candidate expects 3 arguments, 1 provided
   std::string lower_value = std::string(arg);
                                            ^
In file included from C:/rtools40/mingw64/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw64/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:576:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:576:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:572:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:572:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:568:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::initializer_list<_Tp>, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(initializer_list<_CharT> __l, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:568:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::initializer_list<char>'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:541:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str) noexcept
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:541:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::__cxx11::basic_string<char>&&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:529:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:529:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:514:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _CharT* __s, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:514:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const char*'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:499:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const _CharT* __s, size_type __n,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:499:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:481:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:481:7: note:   candidate expects 4 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:465:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:465:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:450:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:450:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:437:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:437:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::__cxx11::basic_string<char>&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:429:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _Alloc& __a) _GLIBCXX_NOEXCEPT
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:429:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::allocator<char>&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:420:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string()
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:420:7: note:   candidate expects 0 arguments, 1 provided
In file included from sentencepiece/src/model_interface.h:31,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h: In function 'bool sentencepiece::string_util::lexical_cast(absl::string_view, Target*) [with Target = std::__cxx11::basic_string<char>]':
sentencepiece/src/util.h:91:28: error: no matching function for call to 'std::__cxx11::basic_string<char>::basic_string(absl::string_view&)'
   *result = std::string(arg);
                            ^
In file included from C:/rtools40/mingw64/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw64/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:614:9: note: candidate: 'template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&)'
         basic_string(_InputIterator __beg, _InputIterator __end,
         ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:614:9: note:   template argument deduction/substitution failed:
In file included from sentencepiece/src/model_interface.h:31,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h:91:28: note:   candidate expects 3 arguments, 1 provided
   *result = std::string(arg);
                            ^
In file included from C:/rtools40/mingw64/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw64/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw64/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:576:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:576:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:572:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:572:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:568:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::initializer_list<_Tp>, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(initializer_list<_CharT> __l, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:568:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::initializer_list<char>'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:541:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str) noexcept
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:541:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::__cxx11::basic_string<char>&&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:529:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:529:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:514:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _CharT* __s, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:514:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const char*'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:499:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const _CharT* __s, size_type __n,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:499:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:481:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:481:7: note:   candidate expects 4 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:465:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:465:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:450:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = long long unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:450:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:437:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str)
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:437:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::__cxx11::basic_string<char>&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:429:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _Alloc& __a) _GLIBCXX_NOEXCEPT
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:429:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::allocator<char>&'
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:420:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string()
       ^~~~~~~~~~~~
C:/rtools40/mingw64/include/c++/8.3.0/bits/basic_string.h:420:7: note:   candidate expects 0 arguments, 1 provided
sentencepiece/src/bpe_model.cc: In member function 'virtual std::vector<std::pair<absl::string_view, int> > sentencepiece::bpe::Model::SampleEncode(absl::string_view, float) const':
sentencepiece/src/bpe_model.cc:39:36: error: 'class absl::string_view' has no member named 'empty'
   if (!status().ok() || normalized.empty()) {
                                    ^~~~~
sentencepiece/src/bpe_model.cc:111:22: error: 'class absl::string_view' has no member named 'empty'
   while (!normalized.empty()) {
                      ^~~~~
sentencepiece/src/bpe_model.cc:116:16: error: 'class absl::string_view' has no member named 'remove_prefix'
     normalized.remove_prefix(mblen);
                ^~~~~~~~~~~~~
sentencepiece/src/bpe_model.cc:117:25: error: 'class absl::string_view' has no member named 'empty'
     s.next = normalized.empty() ? -1 : index + 1;
                         ^~~~~
sentencepiece/src/bpe_model.cc:146:34: error: 'class absl::string_view' has no member named 'empty'
     if (symbols[top->left].piece.empty() || symbols[top->right].piece.empty() ||
                                  ^~~~~
sentencepiece/src/bpe_model.cc:146:71: error: 'class absl::string_view' has no member named 'empty'
     if (symbols[top->left].piece.empty() || symbols[top->right].piece.empty() ||
                                                                       ^~~~~
make: *** [C:/PROGRA~1/R/R-40~1.2/etc/x64/Makeconf:227: sentencepiece/src/bpe_model.o] Error 1
@taku910 taku910 added execution environment Any issues related to execution environment, installation help wanted labels Jan 10, 2021
@xiefangqi
Copy link
Contributor

xiefangqi commented Dec 9, 2021 •
edited

@jwijffels Can this problem be reproduced now? I tested v0.1.84 and v0.1.96 on mingw, both of them can be compiled successfully. Here is my environment:
1、download and install mingw from https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z
2、install cmake 3.14.7
3、copy mingw32-make.exe to make.exe in /your_path/mingw64/bin/

I compile sentencepiece with:

D:\sentencepiece\sentencepiece\build>cmake -G "CodeBlocks - MinGW Makefiles" ..
-- VERSION: 0.1.86
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: D:/mingw64/bin/gcc.exe
-- Check for working C compiler: D:/mingw64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: D:/mingw64/bin/g++.exe
-- Check for working CXX compiler: D:/mingw64/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Not Found TCMalloc: TCMALLOC_LIB-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: D:/sentencepiece/sentencepiece/build

D:\sentencepiece\sentencepiece\build>make -j4
Scanning dependencies of target sentencepiece
Scanning dependencies of target sentencepiece-static
Scanning dependencies of target sentencepiece_train-static
[ 2%] [ 2%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/builder.cc.objBuilding CXX object src/CMakeFiles/sentencepiece_train-static.dir/unicode_script.cc.obj

[ 2%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/arena.cc.obj
[ 3%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/arena.cc.obj
[ 4%] [ 4%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/arenastring.cc.obj
Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/arenastring.cc.obj
[ 5%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/bytestream.cc.obj
[ 6%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/bytestream.cc.obj
[ 7%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/coded_stream.cc.obj
[ 8%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/coded_stream.cc.obj
[ 9%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/common.cc.obj
[ 10%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/common.cc.obj
D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc: In member function 'google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long int)':
D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51: In member function ' google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long int)warning: ':
unknown conversion type character 'D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:l ' in format [warning: -Wformat=unknown conversion type character ']
snprintf(buffer, sizeof(buffer), FORMAT, valuel)' in format [;
-Wformat=^]
snprintf(buffer, sizeof(buffer), FORMAT, value
)D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:246:1:;
^note:
in expansion of macro 'D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:246:1:DECLARE_STREAM_OPERATOR '
note: DECLARE_STREAM_OPERATORin expansion of macro '(long long , "%" GOOGLE_LL_FORMAT "d")
DECLARE_STREAM_OPERATOR^~~~~~~~~~~~~~~~~~~~~~~'

DECLARE_STREAM_OPERATORD:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:(long long , "%" GOOGLE_LL_FORMAT "d")
^~~~~~~~~~~~~~~~~~~~~~~warning:
too many arguments for format [D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:-Wformat-extra-args ]
snprintf(buffer, sizeof(buffer), FORMAT, valuewarning: )too many arguments for format [;
-Wformat-extra-args^]
snprintf(buffer, sizeof(buffer), FORMAT, value
)D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:246:1:;
^note:
in expansion of macro 'D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:246:1:DECLARE_STREAM_OPERATOR '
note: DECLARE_STREAM_OPERATORin expansion of macro '(long long , "%" GOOGLE_LL_FORMAT "d")
DECLARE_STREAM_OPERATOR^~~~~~~~~~~~~~~~~~~~~~~'

DECLARE_STREAM_OPERATOR(long long , "%" GOOGLE_LL_FORMAT "d")
^~~~~~~~~~~~~~~~~~~~~~~D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:
In member function 'google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long unsigned int)D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:':
In member function 'D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:google::protobuf::internal::LogMessage& google::protobuf::internal::LogMessage::operator<<(long long unsigned int) ':
warning: D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:
warning: unknown conversion type character '
snprintf(buffer, sizeof(buffer), FORMAT, value' in format [
]
snprintf(buffer, sizeof(buffer), FORMAT, value
; \

7:1:

ERATOR
in expansion of macro '
'

(unsigned long long, "%" GOOGLE_LL_FORMAT "u")
D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:^~~~~~~~~~~~~~~~~~~~~~~
warning: D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:232:51:too many arguments for format [ -Wformat-extra-argswarning: ]
snprintf(buffer, sizeof(buffer), FORMAT, value )-Wformat-extra-args;
snprintf(buffer, sizeof(buffer), FORMAT, value^)

                                               D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:247:1:^

note: D:\sentencepiece\sentencepiece\third_party\protobuf-lite\common.cc:247:1:in expansion of macro ' DECLARE_STREAM_OPERATORnote: '
DECLARE_STREAM_OPERATORDECLARE_STREAM_OPERATOR(unsigned long long, "%" GOOGLE_LL_FORMAT "u")

^~~~~~~~~~~~~~~~~~~~~~~DECLARE_STREAM_OPERATOR
(unsigned long long, "%" GOOGLE_LL_FORMAT "u")
^~~~~~~~~~~~~~~~~~~~~~~
[ 12%] [ 12%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/extension_set.cc.objBuilding CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/extension_set.cc.obj

[ 13%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/generated_message_table_driven_lite.cc.obj
[ 14%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/generated_message_table_driven_lite.cc.obj
[ 15%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/generated_message_util.cc.obj
[ 16%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/trainer_factory.cc.obj
[ 17%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/generated_message_util.cc.obj
[ 18%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/implicit_weak_message.cc.obj
[ 19%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/implicit_weak_message.cc.obj
[ 20%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/int128.cc.obj
[ 21%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/trainer_interface.cc.obj
[ 22%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/io_win32.cc.obj
[ 23%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/unigram_model_trainer.cc.obj
[ 24%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/int128.cc.obj
[ 25%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/io_win32.cc.obj
[ 26%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/message_lite.cc.obj
[ 27%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/message_lite.cc.obj
[ 28%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/repeated_field.cc.obj
[ 29%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/status.cc.obj
[ 30%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/repeated_field.cc.obj
[ 31%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/statusor.cc.obj
[ 31%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/status.cc.obj
[ 31%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/stringpiece.cc.obj
[ 32%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/word_model_trainer.cc.obj
[ 33%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/statusor.cc.obj
[ 34%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/stringprintf.cc.obj
[ 35%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/stringpiece.cc.obj
[ 36%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/structurally_valid.cc.obj
[ 36%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/char_model_trainer.cc.obj
[ 37%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/stringprintf.cc.obj
[ 38%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/strutil.cc.obj
[ 39%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/structurally_valid.cc.obj
[ 40%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/time.cc.obj
[ 41%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/strutil.cc.obj
[ 42%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/bpe_model_trainer.cc.obj
[ 43%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/wire_format_lite.cc.obj
[ 44%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/time.cc.obj
[ 45%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/sentencepiece_trainer.cc.obj
[ 46%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/wire_format_lite.cc.obj
[ 47%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/zero_copy_stream.cc.obj
[ 48%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/protobuf-lite/zero_copy_stream_impl_lite.cc.obj
[ 49%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/zero_copy_stream.cc.obj
[ 50%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/builtin_pb/sentencepiece.pb.cc.obj
[ 51%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/protobuf-lite/zero_copy_stream_impl_lite.cc.obj
[ 52%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/builtin_pb/sentencepiece_model.pb.cc.obj
[ 53%] Building CXX object src/CMakeFiles/sentencepiece.dir/builtin_pb/sentencepiece.pb.cc.obj
[ 54%] Building CXX object src/CMakeFiles/sentencepiece.dir/builtin_pb/sentencepiece_model.pb.cc.obj
[ 55%] Linking CXX static library libsentencepiece_train.a
[ 55%] Built target sentencepiece_train-static
[ 56%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/bpe_model.cc.obj
[ 57%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/char_model.cc.obj
[ 58%] Building CXX object src/CMakeFiles/sentencepiece.dir/bpe_model.cc.obj
[ 59%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/error.cc.obj
[ 60%] Building CXX object src/CMakeFiles/sentencepiece.dir/char_model.cc.obj
[ 60%] Building CXX object src/CMakeFiles/sentencepiece.dir/error.cc.obj
[ 60%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/filesystem.cc.obj
[ 61%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/flags.cc.obj
[ 62%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/model_factory.cc.obj
[ 63%] Building CXX object src/CMakeFiles/sentencepiece.dir/filesystem.cc.obj
[ 64%] Building CXX object src/CMakeFiles/sentencepiece.dir/flags.cc.obj
[ 65%] Building CXX object src/CMakeFiles/sentencepiece.dir/model_factory.cc.obj
[ 66%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/model_interface.cc.obj
[ 67%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/normalizer.cc.obj
[ 68%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/sentencepiece_processor.cc.obj
[ 69%] Building CXX object src/CMakeFiles/sentencepiece.dir/model_interface.cc.obj
[ 70%] Building CXX object src/CMakeFiles/sentencepiece.dir/normalizer.cc.obj
[ 71%] Building CXX object src/CMakeFiles/sentencepiece.dir/sentencepiece_processor.cc.obj
[ 72%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/unigram_model.cc.obj
[ 73%] Building CXX object src/CMakeFiles/sentencepiece.dir/unigram_model.cc.obj
[ 74%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/util.cc.obj
[ 75%] Building CXX object src/CMakeFiles/sentencepiece.dir/util.cc.obj
[ 76%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/word_model.cc.obj
[ 77%] Building CXX object src/CMakeFiles/sentencepiece.dir/word_model.cc.obj
[ 78%] Building CXX object src/CMakeFiles/sentencepiece-static.dir//third_party/absl/strings/string_view.cc.obj
[ 79%] Building CXX object src/CMakeFiles/sentencepiece.dir//third_party/absl/strings/string_view.cc.obj
[ 80%] Linking CXX static library libsentencepiece.a
[ 80%] Built target sentencepiece-static
[ 81%] Linking CXX shared library libsentencepiece.dll
[ 81%] Built target sentencepiece
Scanning dependencies of target sentencepiece_train
Scanning dependencies of target spm_encode
Scanning dependencies of target spm_export_vocab
Scanning dependencies of target spm_decode
[ 82%] Building CXX object src/CMakeFiles/spm_encode.dir/spm_encode_main.cc.obj
[ 84%] Building CXX object src/CMakeFiles/spm_decode.dir/spm_decode_main.cc.obj
[ 84%] Building CXX object src/CMakeFiles/spm_export_vocab.dir/spm_export_vocab_main.cc.obj
[ 85%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/builder.cc.obj
[ 86%] Linking CXX executable spm_export_vocab.exe
[ 86%] Built target spm_export_vocab
[ 87%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/unicode_script.cc.obj
[ 88%] Linking CXX executable spm_decode.exe
[ 88%] Built target spm_decode
[ 89%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/trainer_factory.cc.obj
[ 90%] Linking CXX executable spm_encode.exe
[ 90%] Built target spm_encode
[ 90%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/trainer_interface.cc.obj
[ 91%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/unigram_model_trainer.cc.obj
[ 92%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/word_model_trainer.cc.obj
[ 93%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/char_model_trainer.cc.obj
[ 94%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/bpe_model_trainer.cc.obj
[ 95%] Building CXX object src/CMakeFiles/sentencepiece_train.dir/sentencepiece_trainer.cc.obj
[ 96%] Linking CXX shared library libsentencepiece_train.dll
[ 96%] Built target sentencepiece_train
Scanning dependencies of target spm_train
Scanning dependencies of target spm_normalize
[ 97%] Building CXX object src/CMakeFiles/spm_train.dir/spm_train_main.cc.obj
[ 98%] Building CXX object src/CMakeFiles/spm_normalize.dir/spm_normalize_main.cc.obj
[ 99%] Linking CXX executable spm_normalize.exe
[ 99%] Built target spm_normalize
[100%] Linking CXX executable spm_train.exe
[100%] Built target spm_train

@xiefangqi
Copy link
Contributor

So i compiled successfully with gcc 7.3, you gcc version was 8.3.0? I think the problem of absl maybe related to the gcc version of mingw.

@jwijffels
Copy link
Author

jwijffels commented Dec 9, 2021 •
edited

The mingw with R ships indeed with gcc-8-3.0 (https://cran.r-project.org/bin/windows/Rtools/rtools40.html)

I'll see that in sentencepiece release v0.1.95 you switched to third_party absl and that you updated the third_party protobuf which contained fixes for the UBSAN issues I reported at #595 and protocolbuffers/protobuf#2073
I tried to incorporate these changes in the R package but that did not work hence this issue here.

If I incorporate the current C++ code (commit 8420f21) I get the same make errors

FYI: compilation makevars is at https://github.com/bnosac/sentencepiece/blob/master/src/Makevars

image

C:\Users\Jan\Dropbox\Work\RForgeBNOSAC\BNOSAC>R CMD INSTALL sentencepiece
* installing to library 'C:/Users/Jan/Documents/R/win-library/4.0'
* installing *source* package 'sentencepiece' ...
** using staged installation
** libs

*** arch - i386
"C:/rtools40/mingw32/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -D HAVE_PTHREAD=1 -pthread -DSTRICT_R_HEADERS -I. -I./sentencepiece -I./sentencepiece/src -I./sentencepiece/src/builtin_pb -I./third_party/absl -I./third_party/darts_clone -I./third_party/esaxx -I./third_party/protobuf-lite  -I'C:/Users/Jan/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c sentencepiece/src/bpe_model.cc -o sentencepiece/src/bpe_model.o
In file included from sentencepiece/src/normalizer.h:27,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
./third_party/absl/strings/string_view.h:152:7: error: redefinition of 'class absl::string_view'
 class string_view {
       ^~~~~~~~~~~
In file included from sentencepiece/src/normalizer.h:26,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/sentencepiece_processor.h:32:7: note: previous definition of 'class absl::string_view'
 class string_view {
       ^~~~~~~~~~~
In file included from sentencepiece/src/normalizer.h:27,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
./third_party/absl/strings/string_view.h:549:58: error: 'npos' is not a member of 'absl::string_view'
                                  size_t n = string_view::npos) {
                                                          ^~~~
./third_party/absl/strings/string_view.h: In function 'absl::string_view absl::ClippedSubstr(absl::string_view, size_t, size_t)':
./third_party/absl/strings/string_view.h:551:12: error: 'class absl::string_view' has no member named 'substr'
   return s.substr(pos, n);
            ^~~~~~
In file included from sentencepiece/src/normalizer.h:29,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h: In member function 'size_t sentencepiece::string_util::string_view_hash::operator()(const absl::string_view&) const':
sentencepiece/src/util.h:65:39: error: no match for 'operator[]' (operand types are 'const absl::string_view' and 'size_t' {aka 'unsigned int'})
       hash = ((hash << 5) + hash) + sp[i];
                                       ^
sentencepiece/src/util.h: In function 'bool sentencepiece::string_util::lexical_cast(absl::string_view, Target*) [with Target = bool]':
sentencepiece/src/util.h:81:44: error: no matching function for call to 'std::__cxx11::basic_string<char>::basic_string(absl::string_view&)'
   std::string lower_value = std::string(arg);
                                            ^
In file included from C:/rtools40/mingw32/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw32/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:614:9: note: candidate: 'template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&)'
         basic_string(_InputIterator __beg, _InputIterator __end,
         ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:614:9: note:   template argument deduction/substitution failed:
In file included from sentencepiece/src/normalizer.h:29,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h:81:44: note:   candidate expects 3 arguments, 1 provided
   std::string lower_value = std::string(arg);
                                            ^
In file included from C:/rtools40/mingw32/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw32/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:576:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:576:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:572:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:572:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:568:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::initializer_list<_Tp>, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(initializer_list<_CharT> __l, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:568:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::initializer_list<char>'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:541:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str) noexcept
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:541:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::__cxx11::basic_string<char>&&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:529:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:529:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:514:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _CharT* __s, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:514:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const char*'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:499:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const _CharT* __s, size_type __n,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:499:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:481:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:481:7: note:   candidate expects 4 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:465:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:465:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:450:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:450:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:437:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:437:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::__cxx11::basic_string<char>&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:429:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _Alloc& __a) _GLIBCXX_NOEXCEPT
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:429:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::allocator<char>&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:420:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string()
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:420:7: note:   candidate expects 0 arguments, 1 provided
In file included from sentencepiece/src/normalizer.h:29,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h: In function 'bool sentencepiece::string_util::lexical_cast(absl::string_view, Target*) [with Target = std::__cxx11::basic_string<char>]':
sentencepiece/src/util.h:99:28: error: no matching function for call to 'std::__cxx11::basic_string<char>::basic_string(absl::string_view&)'
   *result = std::string(arg);
                            ^
In file included from C:/rtools40/mingw32/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw32/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:614:9: note: candidate: 'template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(_InputIterator, _InputIterator, const _Alloc&)'
         basic_string(_InputIterator __beg, _InputIterator __end,
         ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:614:9: note:   template argument deduction/substitution failed:
In file included from sentencepiece/src/normalizer.h:29,
                 from sentencepiece/src/model_interface.h:25,
                 from sentencepiece/src/bpe_model.h:18,
                 from sentencepiece/src/bpe_model.cc:22:
sentencepiece/src/util.h:99:28: note:   candidate expects 3 arguments, 1 provided
   *result = std::string(arg);
                            ^
In file included from C:/rtools40/mingw32/include/c++/8.3.0/string:52,
                 from C:/rtools40/mingw32/include/c++/8.3.0/stdexcept:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/array:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/tuple:39,
                 from C:/rtools40/mingw32/include/c++/8.3.0/functional:54,
                 from sentencepiece/src/bpe_model.cc:15:
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:576:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:576:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:572:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str, const _Alloc& __a)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:572:7: note:   candidate expects 2 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:568:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::initializer_list<_Tp>, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(initializer_list<_CharT> __l, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:568:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::initializer_list<char>'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:541:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(basic_string&& __str) noexcept
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:541:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'std::__cxx11::basic_string<char>&&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:529:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, _CharT, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(size_type __n, _CharT __c, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:529:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:514:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _CharT* __s, const _Alloc& __a = _Alloc())
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:514:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const char*'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:499:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const _CharT* __s, size_type __n,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:499:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:481:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:481:7: note:   candidate expects 4 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:465:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:465:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:450:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::size_type = unsigned int]'
       basic_string(const basic_string& __str, size_type __pos,
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:450:7: note:   candidate expects 3 arguments, 1 provided
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:437:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const basic_string& __str)
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:437:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::__cxx11::basic_string<char>&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:429:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _Alloc&) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string(const _Alloc& __a) _GLIBCXX_NOEXCEPT
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:429:7: note:   no known conversion for argument 1 from 'absl::string_view' to 'const std::allocator<char>&'
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:420:7: note: candidate: 'std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>]'
       basic_string()
       ^~~~~~~~~~~~
C:/rtools40/mingw32/include/c++/8.3.0/bits/basic_string.h:420:7: note:   candidate expects 0 arguments, 1 provided
sentencepiece/src/bpe_model.cc: In member function 'virtual std::vector<std::pair<absl::string_view, int> > sentencepiece::bpe::Model::SampleEncode(absl::string_view, float) const':
sentencepiece/src/bpe_model.cc:39:36: error: 'class absl::string_view' has no member named 'empty'
   if (!status().ok() || normalized.empty()) {
                                    ^~~~~
sentencepiece/src/bpe_model.cc:111:22: error: 'class absl::string_view' has no member named 'empty'
   while (!normalized.empty()) {
                      ^~~~~
sentencepiece/src/bpe_model.cc:116:16: error: 'class absl::string_view' has no member named 'remove_prefix'
     normalized.remove_prefix(mblen);
                ^~~~~~~~~~~~~
sentencepiece/src/bpe_model.cc:117:25: error: 'class absl::string_view' has no member named 'empty'
     s.next = normalized.empty() ? -1 : index + 1;
                         ^~~~~
sentencepiece/src/bpe_model.cc:147:34: error: 'class absl::string_view' has no member named 'empty'
     if (symbols[top->left].piece.empty() || symbols[top->right].piece.empty() ||
                                  ^~~~~
sentencepiece/src/bpe_model.cc:147:71: error: 'class absl::string_view' has no member named 'empty'
     if (symbols[top->left].piece.empty() || symbols[top->right].piece.empty() ||
                                                                       ^~~~~
make: *** [C:/PROGRA~1/R/R-40~1.2/etc/i386/Makeconf:227: sentencepiece/src/bpe_model.o] Error 1

@jwijffels
Copy link
Author

Maybe https://github.com/google/sentencepiece/blob/master/src/CMakeLists.txt#L15:L29 needs to be rewritten to https://github.com/bnosac/sentencepiece/blob/master/src/Makevars

@jwijffels
Copy link
Author

Ok, took me a while today but I found a solution.
I added the -D_USE_INTERNAL_STRING_VIEW flag to the compilation script

I'll do some more testing on sentencepiece 0.1.96 before I'll update the R package on CRAN inculding version 0.1.96

Thanks again.

@jwijffels
Copy link
Author

If uploaded the R package wrapping sentencepiece 0.1.96 to CRAN yesterday (https://cran.r-project.org/package=sentencepiece)
The issues in this issue have been fixed by adding the D_USE_INTERNAL_STRING_VIEW compilation flag at https://github.com/bnosac/sentencepiece/blob/master/src/Makevars

Closing this as this error seems to be solved

@xiefangqi xiefangqi mentioned this issue Dec 21, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
execution environment Any issues related to execution environment, installation help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jwijffels @taku910 @xiefangqi