Compiling ACE on MacOS

trimblet · May 26, 2020, 8:07am

Hi all (esp. @sweaglesw) ,

Apologies if this has an answer in the ether somewhere, but I’m trying to get ACE to compile on macOS (for this, currently on 10.14). I’m not super up on C compilation, but I have learned a lot in the last couple days! However, I have gone long passed 10 minutes with this problem.

So far, I have made progress (I think) on the compilation, and it seems almost everything is happy and most of the files compiled. However, I’m getting a Undefined symbols for architecture x86_64 error for boost, and so I’m wondering if I’m using the wrong version or something?

Progress/steps:

librepp is compiling fine and in /usr/local/lib/
I commented in the MacOSX.config includes for both /Makefile and /post/Makefile
I’m using gcc 9.3.0 and I had to remove -fnested-functions from MacOSX.config, but it seems to work without?
libboost_regex.a is in /usr/local/lib - I’m getting the version Homebrew is giving me, which seems to be 1.72.0

Any advice on moving beyond the boost problem? Anything else I need to worry about?

Full error response:

bash-3.2$ make
gcc -Wl,-no_pie  -g -O6 -fomit-frame-pointer -funsigned-char -falign-loops=32 -funroll-loops  -m64 -DMACOSX_GCC  -I post/ -DPOST  lexicon.o chart.o dag.o type.o tdl.o rule.o morpho.o roots.o freeze.o unify.o qc.o agenda.o net.o glb.o semindex.o hash.o mrs.o mrsvpm.o mrsdg.o itsdb.o pack.o unpack.o maxent.o generate.o parse.o lui.o conf.o preprocessor.o treebank-control.o token.o lattice-mapping.o lexical-parse.o generalize.o transfer.o transfer-result.o edge-vectors.o forest-out.o exunpack.o semilattice.o rebuild-th.o compile-qc.o idiom.o yy.o lisp.o tnt.o tree.o reconstruct.o arbiter.o profiler.o qcparse.o rule-use-model.o ubertag.o dublin.o licenses.o dag-provenance.o semi.o timeout.o csaw/csaw.o csaw/naive.o csaw/normalize.o main.o post/post.o timer.o linenoise.o lui-cli.o -o ace -lutil /usr/local/lib/librepp.a  -lpthread -lm /usr/local/lib/libboost_regex.a -lstdc++ -ldl
rm -f libace.so
gcc -shared pic/lexicon.o pic/chart.o pic/dag.o pic/type.o pic/tdl.o pic/rule.o pic/morpho.o pic/roots.o pic/freeze.o pic/unify.o pic/qc.o pic/agenda.o pic/net.o pic/glb.o pic/semindex.o pic/hash.o pic/mrs.o pic/mrsvpm.o pic/mrsdg.o pic/itsdb.o pic/pack.o pic/unpack.o pic/maxent.o pic/generate.o pic/parse.o pic/lui.o pic/conf.o pic/preprocessor.o pic/treebank-control.o pic/token.o pic/lattice-mapping.o pic/lexical-parse.o pic/generalize.o pic/transfer.o pic/transfer-result.o pic/edge-vectors.o pic/forest-out.o pic/exunpack.o pic/semilattice.o pic/rebuild-th.o pic/compile-qc.o pic/idiom.o pic/yy.o pic/lisp.o pic/tnt.o pic/tree.o pic/reconstruct.o pic/arbiter.o pic/profiler.o pic/qcparse.o pic/rule-use-model.o pic/ubertag.o pic/dublin.o pic/licenses.o pic/dag-provenance.o pic/semi.o pic/timeout.o pic/csaw/csaw.o pic/csaw/naive.o pic/csaw/normalize.o pic/libace.o pic/timer.o post/post.o -Wl,-soname,libace.so -o libace.so -lrepp  -ldl -lutil
ld: unknown option: -soname
collect2: error: ld returned 1 exit status
make: *** [libace.so] Error 1
make: *** Waiting for unfinished jobs....
Undefined symbols for architecture x86_64:
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::compare(unsigned long, unsigned long, char const*, unsigned long) const", referenced from:
      boost::re_detail_107200::lookup_default_collate_name(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in libboost_regex.a(regex_traits_defaults.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::compare(wchar_t const*) const", referenced from:
      wchar_t* boost::re_detail_107200::re_is_set_member<wchar_t*, wchar_t, boost::c_regex_traits<wchar_t>, unsigned int>(wchar_t*, wchar_t*, boost::re_detail_107200::re_set_long<unsigned int> const*, boost::re_detail_107200::regex_data<wchar_t, boost::c_regex_traits<wchar_t> > const&, bool) in libboost_regex.a(wide_posix_api.o)
      wchar_t const* boost::re_detail_107200::re_is_set_member<wchar_t const*, wchar_t, boost::c_regex_traits<wchar_t>, unsigned int>(wchar_t const*, wchar_t const*, boost::re_detail_107200::re_set_long<unsigned int> const*, boost::re_detail_107200::regex_data<wchar_t, boost::c_regex_traits<wchar_t> > const&, bool) in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::compare(unsigned long, unsigned long, wchar_t const*, unsigned long) const", referenced from:
      unsigned int boost::re_detail_107200::find_sort_syntax<boost::c_regex_traits<wchar_t>, wchar_t>(boost::c_regex_traits<wchar_t> const*, wchar_t*) in libboost_regex.a(wc_regex_traits.o)
  "std::__1::__vector_base_common<true>::__throw_length_error() const", referenced from:
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_alt() in libboost_regex.a(wide_posix_api.o)
      void boost::re_detail_107200::named_subexpressions::set_name<wchar_t>(wchar_t const*, wchar_t const*, int) in libboost_regex.a(wide_posix_api.o)
      void std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::__push_back_slow_path<std::__1::pair<unsigned long, unsigned long> >(std::__1::pair<unsigned long, unsigned long>&&) in libboost_regex.a(wide_posix_api.o)
      void std::__1::vector<boost::re_detail_107200::digraph<wchar_t>, std::__1::allocator<boost::re_detail_107200::digraph<wchar_t> > >::__push_back_slow_path<boost::re_detail_107200::digraph<wchar_t> const&>(boost::re_detail_107200::digraph<wchar_t> const&) in libboost_regex.a(wide_posix_api.o)
      std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >::assign(unsigned long, unsigned char const&) in libboost_regex.a(wide_posix_api.o)
      void std::__1::vector<std::__1::pair<bool, boost::re_detail_107200::re_syntax_base*>, std::__1::allocator<std::__1::pair<bool, boost::re_detail_107200::re_syntax_base*> > >::__push_back_slow_path<std::__1::pair<bool, boost::re_detail_107200::re_syntax_base*> >(std::__1::pair<bool, boost::re_detail_107200::re_syntax_base*>&&) in libboost_regex.a(wide_posix_api.o)
      __ZNSt3__16vectorIN5boost9sub_matchIPKwEENS_9allocatorIS5_EEE6assignIPS5_EENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr16is_constructibleIS5_NS_15iterator_traitsISC_E9referenceEEE5valueEvE4typeESC_SC_ in libboost_regex.a(wide_posix_api.o)
      ...
  "std::__1::__vector_base_common<true>::__throw_out_of_range() const", referenced from:
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_open_paren() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_perl_extension() in libboost_regex.a(wide_posix_api.o)
  "std::__1::__basic_string_common<true>::__throw_length_error() const", referenced from:
      _regerrorW in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::fail(boost::regex_constants::error_type, long) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_extended_escape() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_repeat_range(bool) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_set() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_inner_set(boost::re_detail_107200::basic_char_set<wchar_t, boost::c_regex_traits<wchar_t> >&) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::fail(boost::regex_constants::error_type, long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long) in libboost_regex.a(wide_posix_api.o)
      ...
  "std::runtime_error::runtime_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      void boost::re_detail_107200::raise_error<boost::regex_traits_wrapper<boost::c_regex_traits<wchar_t> > >(boost::regex_traits_wrapper<boost::c_regex_traits<wchar_t> > const&, boost::regex_constants::error_type) in libboost_regex.a(wide_posix_api.o)
  "std::runtime_error::runtime_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      boost::regex_error::regex_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, boost::regex_constants::error_type, long) in libboost_regex.a(regex.o)
      boost::regex_error::regex_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, boost::regex_constants::error_type, long) in libboost_regex.a(regex.o)
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::append(char const*)", referenced from:
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::fail(boost::regex_constants::error_type, long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long) in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::append(char const*, unsigned long)", referenced from:
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::fail(boost::regex_constants::error_type, long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long) in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign(char const*)", referenced from:
      _regerrorW in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse(wchar_t const*, wchar_t const*, unsigned int) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::fail(boost::regex_constants::error_type, long) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_extended() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_all() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::unwind_alts(long) in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_extended_escape() in libboost_regex.a(wide_posix_api.o)
      boost::re_detail_107200::basic_regex_parser<wchar_t, boost::c_regex_traits<wchar_t> >::parse_repeat(unsigned long, unsigned long) in libboost_regex.a(wide_posix_api.o)
      ...
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::erase(unsigned long, unsigned long)", referenced from:
      boost::c_regex_traits<wchar_t>::transform(wchar_t const*, wchar_t const*) in libboost_regex.a(wc_regex_traits.o)
      boost::c_regex_traits<wchar_t>::transform_primary(wchar_t const*, wchar_t const*) in libboost_regex.a(wc_regex_traits.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::append(unsigned long, wchar_t)", referenced from:
      boost::c_regex_traits<wchar_t>::transform(wchar_t const*, wchar_t const*) in libboost_regex.a(wc_regex_traits.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::assign(unsigned long, wchar_t)", referenced from:
      wchar_t* boost::re_detail_107200::re_is_set_member<wchar_t*, wchar_t, boost::c_regex_traits<wchar_t>, unsigned int>(wchar_t*, wchar_t*, boost::re_detail_107200::re_set_long<unsigned int> const*, boost::re_detail_107200::regex_data<wchar_t, boost::c_regex_traits<wchar_t> > const&, bool) in libboost_regex.a(wide_posix_api.o)
      wchar_t const* boost::re_detail_107200::re_is_set_member<wchar_t const*, wchar_t, boost::c_regex_traits<wchar_t>, unsigned int>(wchar_t const*, wchar_t const*, boost::re_detail_107200::re_set_long<unsigned int> const*, boost::re_detail_107200::regex_data<wchar_t, boost::c_regex_traits<wchar_t> > const&, bool) in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::insert(std::__1::__wrap_iter<wchar_t const*>, wchar_t)", referenced from:
      boost::re_detail_107200::basic_regex_creator<wchar_t, boost::c_regex_traits<wchar_t> >::append_set(boost::re_detail_107200::basic_char_set<wchar_t, boost::c_regex_traits<wchar_t> > const&, mpl_::bool_<false>*) in libboost_regex.a(wide_posix_api.o)
  "std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >::__grow_by(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long)", referenced from:
      boost::c_regex_traits<wchar_t>::transform_primary(wchar_t const*, wchar_t const*) in libboost_regex.a(wc_regex_traits.o)
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make: *** [ace] Error 1

Note, I also just changed -Wl,-soname,libace.so to -Wl,-install_name,libace.so based on the warning in the above paste for libace.so, but that didn’t seem to change anything.

For reference, full response after clean is here.

sweaglesw · May 26, 2020, 8:25pm

Hi TJ,

I believe that error indicates the standard C++ library is missing from the link. Try -lstdc++ or -lc++ … I can’t recall which right now?

You are right that fnested-functions is to be removed. Apple used to ship a compiler that required that flag to support inner functions, but nowadays their compiler just doesn’t support them at all, so you are right to use GCC — which doesn’t require that option.

Woodley

trimblet · May 26, 2020, 9:46pm

Thanks, @sweaglesw! The ticket seemed to be replacing the existing -lstdc++ with -lc++. I’m guessing this is a clang vs gcc thing?

Anyway, with that ace will be available on homebrew momentarily!

sweaglesw · May 27, 2020, 2:13am

Thanks TJ!

trimblet · March 13, 2021, 9:22pm

@sweaglesw I’m finally catching up on this and updating the homebrew formula with the latest ACE version as well as giving the option to download older versions. However, it seems something has changed in the last 9 months and now I’m unable to build ACE from source, not 0.9.31, the version I was using before, or any of the newer versions (through 0.9.34). Everything proceeds through the build script as expected until the following command where I get this snippet repeated for very many file pairs:

Command:

gcc -Wl,-no_pie  -g -O6 -fno-omit-frame-pointer -funsigned-char -falign-loops=32 -funroll-loops  -m64 -DMACOSX_GCC  -I post/ -DPOST  lexicon.o  chart.o dag.o type.o tdl.o rule.o morpho.o roots.o freeze.o unify.o qc.o agenda.o net.o glb.o semindex.o hash.o mrs.o mrsvpm.o mrsdg.o itsdb.o  pack.o unpack.o maxent.o generate.o parse.o lui.o conf.o preprocessor.o treebank-control.o token.o lattice-mapping.o lexical-parse.o generalize.o transfer.o transfer-result.o edge-vectors.o forest-out.o exunpack.o semilattice.o rebuild-th.o compile-qc.o idiom.o yy.o lisp.o tnt.o tree.o  reconstruct.o arbiter.o profiler.o qcparse.o rule-use-model.o ubertag.o dublin.o licenses.o dag-provenance.o semi.o timeout.o csaw/csaw.o csaw/naive.o csaw/normalize.o main.o post/post.o timer.o linenoise.o lui-cli.o -o ace -lutil /usr/local/lib/librepp.a  -lpthread -lm /usr/local/lib/libboost_regex.a -lc++ -ldl

Error snippet:

duplicate symbol '_robustness_marker_path' in:
    lexicon.o
    chart.o
duplicate symbol '_robustness_marker_type' in:
    lexicon.o
    chart.o

I did some digging, but as far as I can tell, these declarations are introduced in conf.c and seem totally fine? I did try externing them, but that didn’t work. I also tried compiling without conf (and several other files), but as expected that didn’t work either

Here’s the compiler I’m using:

$ gcc --version
gcc-10 (Homebrew GCC 10.2.0_4) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.

I also tried using gcc version 9, but that didn’t change anything.

Any tips?

curtosis · March 14, 2021, 3:21am

I had a minute to tinker so I took a look … I was able to get it to build by making robustness_marker_path and robustness_marker_type extern in conf.h, which kind of makes sense? I then ran into glb_type_count being duplicated, but was able to solve that by just commenting out the definition in type.c (it’s not used in that file) so there’s no interference with semilattice. It at least passes the brew test test.

That said, I can’t answer why it used to build but now doesn’t without those changes.

One addition: I’d recommend also tweaking MacOSX.config to not hardcode /usr/local for BOOST_REGEX_LIBS and REPP_LIBS – using HOMEBREW_PREFIX will allow it to work wherever the homebrew install is (important for non-admin and Apple Silicon* installations, for example).

*NB. It builds on Apple Silicon but crashes on run, as one might expect: ace is very hands-on with memory, and the layout on M1 is very different.

trimblet · March 14, 2021, 3:38am

Thanks @curtosis! I’ll take a look. I’m wary of commenting out the definition without @sweaglesw’s blessing, seems like it might be important for semi processing (totally a guess based on the file name). Also curious why I wasn’t running into this issue before.

Thanks so much for trying it out on the M1! I’ve been hoping someone would give it a go and report it as a bug since I don’t have one yet, so kudos to you I do agree that it’s very possible that there may need to be some more thorough work that @sweaglesw may or may not want to do with the ARM transition. When I get my hands on an ARM machine, I’ll give it a go!

trimblet · March 14, 2021, 3:44am

Also, @curtosis if you want to try to get it to compile on M1 and/or contribute updates to the formula, please feel free to do so!

curtosis · March 14, 2021, 3:47am

Definitely – I just tried at it as a diagnostic based on what I think should be going on. I make no claims as to its actual correctness.

I haven’t dug too deeply into it yet, so I don’t know how much work is involved. I do know that the SBCL port (for example) needed some work on its memory map, for the same sorts of reasons. I should at least be able to get to the “oh, this is definitely @sweaglesw territory” point.

FWIW, It’s hands-down the best computer I’ve ever owned. (Base model MBA.)

curtosis · March 14, 2021, 3:48am

I ought to be able to do a quick PR for the formula updates ~~tomorrow~~ when @sweaglesw blesses the fixes.

sweaglesw · March 14, 2021, 5:07am

As you suspected, the duplicate definitions of robustness_marker_type, robustness_marker_path, and glb_type_count are all unintentional, and the fixes you proposed are very reasonable. The functionality enabled by those variables is quite obscure, so off hand it would be hard to say whether released versions may have exhibited improper behavior related to those latent bugs under some compilers. Some compilers (I haven’t looked into which ones behave how, but at minimum including the gcc homebrew 9.2.0 I have on this Mac…) automatically merge such multiply defined variables, which in this case would have been the intended behavior.

I don’t yet have access to an M1 to develop on, so I can’t say much about what might be needed to make ace work on it, but I’m not aware of anything that should make it impossible. @curtosis, does it crash while compiling the new grammar image, or while attempting to parse with it? Does it print anything interesting?

trimblet · March 14, 2021, 5:31am

Awesome! Thanks for the info, Woodley! I was able to get the changes @curtosis suggested working, so I went ahead and merged my changes into the repo, bumping the main version to 0.9.34, adding formulae for 0.9.31-33, externing robustness_marker_(path|type), and commenting out the glb_type_count definition! Thanks!

Chris, I can make a separate PR for changing the MacOSX.config, or if you want to, go for it!

curtosis · March 14, 2021, 11:05pm

Thanks for the confirmation, @sweaglesw ! So far it’s failing on unfreezing the compiled grammar:

mmap returned 0x7000000000
mmap didn't put the freezer where I wanted it

FREEZER_MMAP_BASE is defined as 0x6000000000, and it fails when mmap doesn’t give that base back. (I tried MAP_FIXED, but mmap doesn’t honor that and still returns 0x7000000000.) I haven’t dug any deeper yet into why that particular base matters; I did try setting it to 0x7000000000 and that gets far enough to segfault, which isn’t terribly surprising – I assume you chose it for a reason.

I do know that page and cacheline size are different on M1, if that factors into the computation (it should probably matter to the slab allocator cache alignment code, at least, but that should only be a performance issue).

I’ll continue digging, but hopefully that might give you some insight into where the ace code makes assumptions about memory.

curtosis · March 15, 2021, 12:12am

I see you’ve already merged the PR I came here to flag for you.

sweaglesw · March 15, 2021, 3:00am

Ok, interesting. Off hand I can’t think of a reason why 0x7000000 is worse than 0x6000000. The important thing is just that it is mapped at a predictable place, i.e. the same place during grammar compilation and every time it is loaded for parsing. After switching to 0x7000000 did you do a clean build and also recompile the grammar image before trying to parse? A segfault is definitely expected if the grammar was compiled at 0x6000000 but loaded for parsing at 0x7000000.

sweaglesw · April 2, 2021, 7:25am

An update on using ACE with the new Apple chips. I’ve got one now and I’ve tried it. The clang that Apple supplies does not support nested functionsl, and while GCC thinks it knows how to do them, it doesn’t actually – it tries to use a trampoline, same as on other architectures, but the stack is system-wide non-executable under this chip, so it always results in a crash. It’s known issue #26 on the gcc-darwin-arm64 github project. There is no timeline for fixing it, although hopefully they will some time. I tried a gcc-10 from homebrew and also built a gcc-11 from the gcc-darwin-amd64 project and both had the same issue.

However, it is possible to run ACE on the M1 under Rosetta. The binaries I distribute on my website don’t work – the new OS wants to allocate the freezer at 0x7000000000 as @curtosis reported above (Chris, was that error actually from a native compiled ACE? I got one to compile but it wouldn’t compile a grammar image for me; I didn’t bother trying to use it with the bundled x86_64 grammar images, but maybe that is what you tried?). However, building an x86_64 OSX ACE binary on another machine with the freezer expectation set to 0x7000000000 and copying it to the M1 machine results in what seems to be a fully functional ACE – albeit surely not as fast as a native one would be. Still, it is 20% faster running on the M1 MacBook Air under Rosetta than it is on the MacBook Pro I compiled it on, which is a 2019 2.4 GHz Core i9 – single threaded comparison. If anyone with an M1 wants to be able to use ACE, I put a test binary here:

http://sweaglesw.org/linguistics/ace-0.9.34-m1-test

You’ll have to $ chmod +x ace-0.9.34-m1-test after downloading it, and you’ll have to compile any grammar image you want to use manually rather than using one you’ve previously compiled or getting an ERG image from my website, but other than that it should work. Let me know if your mileage varies.

curtosis · April 3, 2021, 10:16pm

Thanks for the update!

Interesting… the error I was seeing is indeed from a native compiled ace. It looks like, though, that what I thought was on unfreeze was actually on freeze (the same error message is in both places). So it looks like Rosetta is the only option until GCC gets fixed; the silver lining is that Rosetta is really good (the special CPU instructions they added to the chip help a lot).

sweaglesw · April 4, 2021, 5:47am

My native compiled ace doesn’t run long enough to get to the point where it is going to freeze a grammar. Nested functions are used pretty liberally, and I get a crash from trying to call one after just 3 lines of output when trying to build the ERG image.

sweaglesw · April 8, 2021, 9:49pm

In case anyone cares, I found a kludge to make ACE work natively on the M1. There’s a special provision for JIT compilers to be allowed to flip a page back and forth between writeable and executable quickly, although it still can’t be both at the same time. I make a ghost copy of the stack set up in JIT mode, and whenever I need to pass the address of an inner function I use a wrapper function that copies the trampoline onto the ghost stack and makes it executable. The impact on the ACE code is quite minimal and it seems to work fine (at least after I realized I also needed to explicitly invalidate the instruction cache…). Compared to the Rosetta version running on M1, it is an additional 20% faster (so about 35% faster compared to the 2019 Intel-based MacBook Pro). I consider this a sort of ugly solution, and I’m not sure it’s worth making a release out of it. Presumably at some point the GCC developers will find a way to make it unnecessary. In the meantime if someone has a compelling reason to need to run ACE natively on M1 (as opposed to using Rosetta with the above-referenced patched ACE binary), let me know…

trimblet · April 9, 2021, 2:07pm

If I had an M1 Mac, I would care a lot!
35% faster seems like a win to me, but it sounds like you would then have two different source releases, which seems undesirable.

fwiw I had tried installing ACE on my raspberry pi six months ago or so in an attempt to see if I could prepare for ARM Macs, but I think I ran into the same GCC problem.