With a few lines you can setup programming environments (compilers, packages, databases, browsers, and other tools from different ecosystems) with having only the Nix Package Manager installed.

You check this environment into source control, share it with colleagues and bring them up to speed in seconds. No manually installing apps and—if gone all the way—no “It works on my machine!” due to different package versions.
More: How to install Nix

When you install Nix, you get the terminal app nix-shell, which can create the following:

  1. Ad hoc environments to quickly try out apps without installing them permanently in your system.
  2. Persistable environments configured with a config file that we can check into Git. Here we can achieve reproducible builds.

In this tutorial we will see how to interact with nix-shell to create both.

Try Out Apps and Tools Within Nix Shell

With Nix installed and a project folder in place

~$ mkdir myproject && cd myproject

you can try out apps from several ecosystem like Javascript, Rust and databases with a single terminal line:

~/myproject$ nix-shell -p nodejs cargo dbeaver firefox
Element Note
~/myproject The folder we are in.
$ Interactive shell prompt symbol allowing us to type terminal commands.
nix-shell Terminal app from the Nix package manager. It creates temporary shell environements. ➡ Examples
-p Flag for nix-shell. Short for --packages. It makes the following Nix packages available in an environment.
nodejs Nix package name for the Javascript build tool npm. ➡ Official website.
cargo Nix package name for the Rust build tool cargo. ➡ Official website.
dbeaver Nix package name for the SQL database management app DBeaver. ➡ Official website.

This downloads the desired packages and places us in a temporary shell environment where the apps like npm and cargo are available:

[nix-shell:~/myproject]$ npm --version
6.14.10

Notice how the shell prompt changes to [nix-shell:~/myproject]$. Running

[nix-shell:~/myproject]$ exit

brings us back into our normal shell where Node and Rust don’t exist:

~/myproject$ npm
npm: command not found

These temporary environments don’t modify or pollute the rest of the system. This is especially useful, when we work on multiple projects that use different versions of apps. These versions will never come into conflict, because they only exist locally in separate, temporary environments.
More: How to search for Nix package names of apps
See: Nix packages source code

info

We can run one-off commands without manually entering and leaving the Nix shell:

$ nix-shell -p nodejs --run 'npm --version'
6.14.10

Put Programming Environments Into Source Control With Nix Shell Files

In most projects we need a lot of tools. Typing out their names into a nix-shell -p ... command every time quickly becomes tedious. Instead we can put all our desired apps and tools into a shell.nix file in the project’s root folder:

# ~/myproject/shell.nix file
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    nodejs
    cargo
    python3
    firefox
    git
    dbeaver # Database GUI
    ripgrep # grep alternative
    vscode # Visual Studio Code
  ];
}
let .. in ..
Define custom variables in the let-part to use in the in-part for more readable code. Without “let” we would have to write (import <nixpkgs> {}).mkShell { ... ➡ Let explanation.
pkgs
Custom variable for the huge collection of all available Nix packages (more than 80000). Think of it as a large Json object { nodejs:.., cargo:.., git:.. } of packages.
import <nixpkgs> {}
Load all available Nix packages ➡ Import explanation.
mkShell
A helper function to create a Nix package that only represents a shell environment. ➡ mkShell explanation.
nativeBuildInputs
List of Nix packages that you want available in the shell environment.
with pkgs; [nodejs git]
Short Nix notation for [pkgs.nodejs pkgs.git] for readability.
[nodejs git]
A list in the Nix language. Nix doesn’t use commas in between!

This file is written in the Nix (programming) language and creates the same shell environment as a nix-shell -p git ... command when we run:

~/myproject$ nix-shell

If you don’t pass any arguments to nix-shell, it uses the shell.nix file in the current folder by convention.
More: Learn the Nix language basics
More: Learn about mkShell and derivations
More: How to search for Nix package names of apps

Let Nix Manage Programming Dependencies As Well

With simple environment definitions as in the previous section like

# shell.nix
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    python3 
    python3Packages.pip 
  ];
}

we let Nix manage only the “top-level” apps and tools. In this example this means that Nix downloads the Python interpreter and the Pip Python package manager from the Nix servers, but Pip would select and download (=manage) Python packages from the Python Pypi servers.

As usual in the Python ecosystem we would write a requirements.txt file listing all the desired Python packages and let Pip download them with

~/myproject$ pip install -r requirements.txt

However, from a Nix persepective with the goal of reproducibility in mind this isn’t optimal: First, we still have to install Python packages manually in a separate step. Second and more importantly, with Pip we can only specify Python package version numbers, but we get no guarantee that our colleagues will get exactly the same source code when they run the Pip command some weeks later. Here, Nix is much more reliable, because it aims to provide bit-identical packages everytime!

info

For serious reproducibility and bit-identical packages in Nix we have to use helper tools like Niv until Nix flakes are stable.

So, to achieve more convenience and reproducibility we can let Nix manage Python packages as well:

# shell.nix
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    python3.withPackages (pyPkgs: with pyPkgs; [ numpy scipy matplotlib notebook scikitlearn scipy nltk spacy])
  ];
}

This shell file example is equivalent to

# Fast.ai jupyter notebook env
$ nix-shell -p 'python3.withPackages(pyPkgs: with pyPkgs; [ numpy scipy matplotlib notebook scikitlearn scipy nltk spacy])'

and sets up an Jupyter Notebook environment that is ready for the fast.ai Deep Learning tutorials.

let .. in ..
Define custom variables in the let-part to use in the in-part for more readable code. Without “let” we would have to write (import <nixpkgs> {}).mkShell { ....
pkgs
Custom variable for the large collection of all available Nix packages (more than 80000). Think of it as a huge Json object { python3:.., git:.. } of packages.
import <nixpkgs> {}
Load all available Nix packages. ➡ Detailed explanation.
mkShell
A helper function to create a Nix package that only represents a shell environment.
nativeBuildInputs
List of Nix packages that you want available in the shell environment.
python3.withPackages
Helper function to bundle the Python3 interpreter up with the following packages. It expects a function as an argument that selects a list of desired Python packages. This precisely what (pyPkgs: with pyPkgs; [ numpy scipy ...]) does: This is a function that receives a list of Python packages to choose from (pyPkgs), and returns a list with the chosen ones. ➡ Nix function syntax.
pyPkgs
Custom variable for the large collection all Python3 packages available in Nix. Think of it as a huge Json object { numpy:.., scipy:.. } of packages. It’s equivalent to the packages in pkgs.python3.pkgs.
with pyPkgs; [numpy scipy]
Short Nix notation for [pyPkgs.numpy pyPkgs.scipy] for readability.
[numpy scipy]
A list in the Nix language; no commas in between!

If you now run

~/myproject$ nix-shell

Nix downloads Python packages from the Nix servers as well, and puts you into a temporary shell environment where the Python interpreter is already bundled up with those packages. No other manual step is required.
More: Learn about mkShell and Nix derivations

info

Nix is able to manage packages from the Python ecosystem, because some Nix-maintainers adopt the most important Python packages as Nix packages. We can find them inside import <nixpkgs> {} (=the huge attribute set containing all Nix packages) as follows:

# set of all Nix packages
{
  git = ....
  ...
  nodejs = ...
  nodePackages = ...
  ...
  haskellPackages = ...
  ...
  # python3 interpreter
  python3 = {
    ...
    # helper function
    withPackages = ...
  };
  # adopted python3 packages
  python3Packages = {
    numpy = ...
    scipy = ...
    ...
  };
  ...
}

As we can see, Nix can manage tools and packages from many different ecosystems: Not only Python, but Javascript, Haskell, Java, Scala and many more.

Tags: nix shell programming environment setup python beginner

Malte Neuss

Java Software Engineer by day, Haskell enthusiast by night.

Other Posts In Series

Just Enough Nix For Programming

The Nix Language for the Functionally Lazy OO Programmer

Nix comes with the powerful, functional Nix (programming) language to manage packages of many ecosystems at once; not just Javascript or Java like Node and Maven do. It’s like writing a declarative package.json or pom.xml file but on steroids: Besides dependency management we get flexible, reproducible development environments for free.

Read More

Just Enough Nix For Programming

How to Search For Apps and Tools within Nix Packages

Nix is very capable of managing dependencies and creating programming environments. Everything is bundled up as Nix packages. However, when we look for an app in a specific version, it can be difficult to find the right Nix package name.

Read More

Just Enough Nix For Programming

Declarative App Builds and Environments: How to create Nix Derivations

Derivations are recipes to build and distribute apps of any ecosystem in Nix. They are similar to Docker files but better, because the Nix programming language permits functions like mkDerivation and mkShell that hide complex details.

Read More