py-pilecore

Public python SDK for the CEMS PileCore web-API

This repository is created by CEMS BV and is a public python wrapper around the CEMS PileCore web-API.

Installation

To install a package in this repository run:

$ pip install py-pilecore

ENV VARS

To use py-pilecore add the follow ENV vars to your environment. Or provide them when asked.

* NUCLEI_TOKEN
    - Your NUCLEI user token

You can obtain your NUCLEI_TOKEN on NUCLEI. Go to personal-access-tokens and create a new user token.

Contribution

Environment

We recommend developing in Python3.11 with a clean virtual environment (using virtualenv or conda), installing the requirements from the requirements.txt file:

Example using virtualenv and pip to install the dependencies in a new environment .env on Linux:

python -m venv .env
source .env/bin/activate
python -m pip install --upgrade pip setuptools
pip install -r requirements.txt
pip install -e .

Documentation

Build the docs:

python -m pip install --upgrade pip setuptools
pip install -r requirements.txt
pip install .

sphinx-build -b html docs public

Format

We format our code with black and isort.

black --config "pyproject.toml" src/pypilecore tests notebooks
isort --settings-path "pyproject.toml" src/pypilecore tests notebooks

Lint

To maintain code quality we use the GitHub super-linter.

Reproduce the full CI lint job (Docker)

The CI lint job runs the super-linter Docker image. To reproduce it exactly, run the run_super_linter.sh bash script from the root directory (requires Docker):

./run_super_linter.sh

Like CI, this lints only the files changed against main and auto-fixes black/isort formatting in place.

Run the Python linters without Docker

The active Python linters are pinned in the lint optional-dependency group. CI lints only the Python files changed against main (and excludes tests/), so collect that file list first, then run each linter against it:

pip install -e ".[lint]"
FILES=$(git diff --name-only main...HEAD -- '*.py' | grep -v '^tests/')

black --check --config "pyproject.toml" $FILES
isort --check-only --settings-path "pyproject.toml" $FILES
flake8 --config ".flake8" $FILES

mypy needs a caveat: super-linter runs it without installing the project, so unresolved third-party imports (matplotlib, numpy, pandas) become Any and their errors disappear. Reproduce that by running mypy from an environment that has only mypy installed:

python -m venv .mypy-venv
.mypy-venv/bin/pip install mypy==2.1.0
.mypy-venv/bin/mypy --config-file "pyproject.toml" --no-install-types $FILES

Running mypy in a fully-installed environment reports extra import-related errors that CI does not, so the Docker script above remains authoritative.

UnitTest

Test the software with the use of coverage:

python -m pip install --upgrade pip setuptools
pip install -r requirements.txt
pip install -e .
coverage run -m pytest

Requirements

Requirements are autogenerated by the pip-compile command with python 3.11

Install pip-tools with:

pip install pip-tools

Generate requirements.txt file with:

pip-compile --extra=test --extra=lint --extra=docs --output-file=requirements.txt pyproject.toml

Update the requirements within the defined ranges with:

pip-compile --upgrade --extra=test --extra=lint --extra=docs --output-file=requirements.txt pyproject.toml

Note that pip-compile might run from your global path. You can also invoke the compile command from the piptools module on a specific python installation (e.g. within a virtualenv):

python -m piptools compile