Creating Young People's First pip Package from Scratch

This article is transcoded by 简悦 SimpRead, original article at sspai.com

Recently, with the help of the Rime Ice Pinyin Project, I dove into Rime, which has now become an indispensable software in my daily life. However, a significant inconvenience when using Rime is the issue with dictionaries. Although there is already the Deep Blue Dictionary Converter supporting conversion between dictionaries, some specific cases still require special handling.

For example, I couldn’t find a professional English medical dictionary, so I had to crawl some terminology for processing and create my own dictionary. To solve this kind of problem, I used my limited Python skills to write a Jupyter notebook, but every time I use it, I have to open VSCode - open the file - select the Python environment - run Jupyter.

It’s really exhausting :pensive_face:

So, can I make it a command-line tool?

I really like bilix, which is a very convenient command-line tool. Just open the terminal, type a few letters, let the terminal autocomplete the rest, paste the link, press Enter, and usually it takes less than 3 seconds.

So I just went for it, opened bili University to search for argparse (the argparse package was used in the earliest Git version of bilix I reviewed; alternatively, you could use click), VSCode! Launch!

After some fiddling, I finally got it running locally:

python rimetool --input-path sample_input/vcf_sample.vcf --tool vcf

It’s just too elegant!

Can I turn it into a pip package?

Again, I really like bilix, which supports pip installation. Just open terminal, type pip install bilix, and you can use it immediately. Compared with downloading my source code from GitHub and then extracting it manually, it’s much more convenient.

So I went back to bili University for further study, and after some trial and error, I finally uploaded rimetool to PyPI. Now you can use it very elegantly:

pip install rimetool

rimetool --input-path sample_input/vcf_sample.vcf --tool vcf

How to create a young person’s first pip package

After the above preparation, you probably have a general idea of how to make a pip package.

  • Write the source code and implement the features
  • Add command-line support
  • Create the pip package and upload it to PyPI

This article is limited in length. I am not good at the first step due to my limited Python skills, and second step command-line tutorials are already quite complete, so I skip these two parts. For the third step, I am not very satisfied with existing online guides but it is not too difficult, so I dare to write a tutorial here.

Apply for an account and get the token

First, go to the PyPI official website to apply for an account (it’s quite simple, so I won’t elaborate).

Then, in the account management page, apply for an API token.

Pick a nice name, select “entire account”. This token has the highest permission, not recommended for daily use. After successfully creating your project, you can create a project-specific token dedicated to handling this project.

Save the generated token in a text editor (it only appears once; if you close this page, it won’t appear again). Now let’s configure the local environment’s token.

For macOS, create a new file $HOME/.pypirc, e.g. /Users/whitewatercn/.pypirc

Edit it with a text editor or tools like vim and save it:

[pypi]
  username = __token__
  password = the token you applied above

Now let’s start compiling!

Go to your project root folder and configure setup.py. This file contains many pip package–related settings. Here’s mine as an example:

from setuptools import setup, find_packages

setup(
    # Required parameters
    name='rimetool',  # module name
    version='0.1.2',  # current version
    description='rime input method related tools',  # short description
    packages=find_packages(include=['rimetool', 'rimetool.*']),  # include rimetool and all subpackages

    # Optional parameters
    long_description="rime input method related tools",  # long description
    url='https://github.com/whitewatercn/rimetool',  # homepage link
    author='whitewatercn',  # author name
    author_email='whitewatercn@outlook.com',  # author email
    classifiers=[
        'Intended Audience :: Developers',  # target audience
        'Topic :: Software Development :: Build Tools',  # topic tags

    ],
    keywords=['rime','input method editor tool','python'],  # keywords separated by spaces
    install_requires=['pypinyin',
                      'argparse',
                      'click'],  # dependencies
    python_requires='>=3.0',  # supported Python version
    entry_points={  # create terminal commands and link to module functions
        'console_scripts': [
            'rimetool=rimetool.main:main',
        ],
        },
        project_urls={  # additional project-related links
        'Bug Reports': 'https://github.com/whitewatercn/rimetool/issues',
        'Source': 'https://github.com/whitewatercn/rimetool',
    },
)

Some of these are mandatory, some aren’t. You can try deleting some content. Special notes:

  • If your project references your own created package, configure packages=find_packages(include=['rimetool', 'rimetool.*'])
  • If you reference other pip packages, configure install_requires=['pypinyin','argparse','click']
  • ~If you encounter import errors, honestly I don’t know either; I resolved mine by rewriting with GPT :dog_face:~

After configuration, you can start compiling from where setup.py is located. Before that, you need to install the required compilation tools twine and wheel:

pip install wheel twine

Then execute:

python setup.py sdist bdist_wheel

This will generate many compiled files in your working directory. If you want to upload source code to GitHub, remember to edit .gitignore to ignore them.

(torch) (base) whitewatercn@MadBookAir rimetool % tree -L 1
.
├── LICENSE
├── README.md
├── rimetool
├── sample_input
└── setup.py

(torch) (base) whitewatercn@MadBookAir rimetool % python setup.py sdist bdist_wheel
(torch) (base) whitewatercn@MadBookAir rimetool % tree -L 1                        
.
├── LICENSE
├── README.md
├── build
├── dist
├── rimetool
├── rimetool.egg-info
├── sample_input
└── setup.py

Before uploading to PyPI, I strongly suggest you run locally first to see if it works as expected:

pip install -e .

This command installs your project locally with editable mode, so edits on source code update in real time. When satisfied, we start uploading!

The last step, upload!

After compiling the latest source code, execute in the path where setup.py is located:

twine upload dist/*

This will automatically use the token in $HOME/.pypirc created previously. If all goes well, the upload will succeed!

You can search for your project at the PyPI official site.

You might encounter some issues; you can check the details with the following command, which gives very detailed error messages:

twine upload dist/* --verbose

For example, I tried to re-upload once just now and it reminded me that this version has been uploaded “File already exists.

(torch) (base) whitewatercn@MadBookAir rimetool % twine upload dist/* --verbose
INFO     Using configuration from /Users/whitewatercn/.pypirc                                                                                                                                                             
Uploading distributions to https://upload.pypi.org/legacy/
INFO     dist/rimetool-0.1.2-py3-none-any.whl (16.0 KB)                                                                                                                                                                   
INFO     dist/rimetool-0.1.2.tar.gz (16.3 KB)                                                                                                                                                                             
INFO     password set from config file                                                                                                                                                                                    
INFO     username: __token__                                                                                                                                                                                              
INFO     password: <hidden>n>
Uploading rimetool-0.1.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.0/20.0 kB • 00:00 • 57.1 MB/s
INFO     Response from https://upload.pypi.org/legacy/:                                                                                                                                                                   
         400 File already exists. See https://pypi.org/help/#file-name-reuse for more information.                                                                                                                         
INFO     <html>                                                                                                                                                                                                           
          <head>                                                                                                                                                                                                          
           <title>400 File already exists. See https://pypi.org/help/#file-name-reuse for more information.</title>                                                                                                       
          </head>                                                                                                                                                                                                         
          <body>                                                                                                                                                                                                          
           <h1>400 File already exists. See https://pypi.org/help/#file-name-reuse for more information.</h1>                                                                                                             
           The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>                                                                                                     
         File already exists. See https://pypi.org/help/#file-name-reuse for more information.                                                                                                                             
                                                                                                                                                                                                                           
                                                                                                                                                                                                                           
          </body>                                                                                                                                                                                                         
         </html>                                                                                                                                                                                                          
ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/                                                                                                                                                  
         File already exists. See https://pypi.org/help/#file-name-reuse for more information.  

Based on my recent experience, the most common issues are naming and version problems

  • pip packages cannot have duplicate names
  • The version='' in the new version’s setup.py must be greater than the previous version

Finally

Wishing all young developers to soon have their first pip package

Also welcome friends who use rime to use & improve rimetool

Related links

rimetool—some rime-related tools

rime_clinic—Chinese-English medical dictionary for rime

Deep Blue dictionary conversion

Mist Pinyin—rime’s long-maintained simplified Chinese dictionary