[Repost] Using pip to locally install kenlm on macOS

🛠 Tools and Programming
1

This article is transcoded by 简悦 SimpRead, original address blog.you7n.com

KenLM is an efficient n-gram language model library, but encountered various issues when installing on Mac. After working through the night and morning, it was finally installed. This article records the method for successfully installing KenLM.

12 Feb 2025 •

KenLM is an efficient n-gram language model library, but encountered various issues when installing on Mac. After working through the night and morning, it was finally installed. This article records the method for successfully installing KenLM.

If you use Python version 3.10 or above, this solution might work for you. For other fixes, see:

When trying to install kenlm using pip or pip3, it is very likely to fail. According to issues on GitHub, it seems there is some problem with the pip kenlm package (?)

Running pip3 install kenlm or pip3 install pypi-kenlm will give the following error:

Mainly showing:

Failed to build kenlm
ERROR: Failed to build installable wheels for some pyproject.toml based projects (kenlm)

So try to manually compile the GitHub version.

git clone https://github.com/kpu/kenlm

Use cd to enter the kenlm folder.

KenLM cannot compile properly under the latest Python 3.13 version, so use Python 3.10.

You can install it via brew:

Recreating the python3 symlink or setting the global Python version is too troublesome. Just install the local library directly via pip3.10.

Since you have already cd into the kenlm folder, install this folder directly:

At this point, the installation should be successful and you can test it. The kenlm/python/example.py is a test file, run it with python3.10:

python3.10 /Users/■■■/kenlm/python/example.py

If the result looks like the picture, it was successful:

That’s all.

2

Cannot install, gave up

Switched to open-ngram

farcompilestrings --symbols=syms.txt --keep_symbols=1 input.txt > input.far

ngramcount --order=3 input.far > input.cnts

ngrammake --method=kneser_ney input.cnts > output.lm

ngramprint --ARPA output.lm > output.arpa
3

Thanks to the blogger for reposting! ^ ^

1 Like
4

Thanks to the original author for the support (✪ω✪)

In the end, I installed the pip package, and it was indeed possible to import kenlm, but I couldn’t perform related operations via the command line.

The command line tools should require compilation to work, so I switched to open-ngram to complete the related tasks.