How to Setup Programming Environments with Nix Shell

With a few lines you can setup programming environments (compilers, packages, databases, browsers, and other tools from different ecosystems) with having only the Nix Package Manager installed.

You check this environment into source control, share it with colleagues and bring them up to speed in seconds. No manually installing apps and—if gone all the way—no “It works on my machine!” due to different package versions.
More: How to install Nix

When you install Nix, you get the terminal app nix-shell, which can create the following:

1. Ad hoc environments to quickly try out apps without installing them permanently in your system. ➡
2. Persistable environments configured with a config file that we can check into Git. Here we can achieve reproducible builds. ➡

In this tutorial we will see how to interact with nix-shell to create both.

Try Out Apps and Tools Within Nix Shell

With Nix installed and a project folder in place

~$ mkdir myproject && cd myproject

you can try out apps from several ecosystem like Javascript, Rust and databases with a single terminal line:

~/myproject$ nix-shell -p nodejs cargo dbeaver firefox

This downloads the desired packages and places us in a temporary shell environment where the apps like npm and cargo are available:

[nix-shell:~/myproject]$ npm --version
6.14.10

Notice how the shell prompt changes to [nix-shell:~/myproject]$. Running

[nix-shell:~/myproject]$ exit

brings us back into our normal shell where Node and Rust don’t exist:

~/myproject$ npm
npm: command not found

These temporary environments don’t modify or pollute the rest of the system. This is especially useful, when we work on multiple projects that use different versions of apps. These versions will never come into conflict, because they only exist locally in separate, temporary environments.
More: How to search for Nix package names of apps
See: Nix packages source code

$ nix-shell -p nodejs --run 'npm --version'
6.14.10

Put Programming Environments Into Source Control With Nix Shell Files

In most projects we need a lot of tools. Typing out their names into a nix-shell -p ... command every time quickly becomes tedious. Instead we can put all our desired apps and tools into a shell.nix file in the project’s root folder:

# ~/myproject/shell.nix file
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    nodejs
    cargo
    python3
    firefox
    git
    dbeaver # Database GUI
    ripgrep # grep alternative
    vscode # Visual Studio Code
  ];
}

This file is written in the Nix (programming) language and creates the same shell environment as a nix-shell -p git ... command when we run:

~/myproject$ nix-shell

If you don’t pass any arguments to nix-shell, it uses the shell.nix file in the current folder by convention.
More: Learn the Nix language basics
More: Learn about mkShell and derivations
More: How to search for Nix package names of apps

Let Nix Manage Programming Dependencies As Well

With simple environment definitions as in the previous section like

# shell.nix
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    python3 
    python3Packages.pip 
  ];
}

we let Nix manage only the “top-level” apps and tools. In this example this means that Nix downloads the Python interpreter and the Pip Python package manager from the Nix servers, but Pip would select and download (=manage) Python packages from the Python Pypi servers.

As usual in the Python ecosystem we would write a requirements.txt file listing all the desired Python packages and let Pip download them with

~/myproject$ pip install -r requirements.txt

However, from a Nix persepective with the goal of reproducibility in mind this isn’t optimal: First, we still have to install Python packages manually in a separate step. Second and more importantly, with Pip we can only specify Python package version numbers, but we get no guarantee that our colleagues will get exactly the same source code when they run the Pip command some weeks later. Here, Nix is much more reliable, because it aims to provide bit-identical packages everytime!

So, to achieve more convenience and reproducibility we can let Nix manage Python packages as well:

# shell.nix
let
  pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs; [
    python3.withPackages (pyPkgs: with pyPkgs; [ numpy scipy matplotlib notebook scikitlearn scipy nltk spacy])
  ];
}

# Fast.ai jupyter notebook env
$ nix-shell -p 'python3.withPackages(pyPkgs: with pyPkgs; [ numpy scipy matplotlib notebook scikitlearn scipy nltk spacy])'

If you now run

~/myproject$ nix-shell

Nix downloads Python packages from the Nix servers as well, and puts you into a temporary shell environment where the Python interpreter is already bundled up with those packages. No other manual step is required.
More: Learn about mkShell and Nix derivations

# set of all Nix packages
{
  git = ....
  ...
  nodejs = ...
  nodePackages = ...
  ...
  haskellPackages = ...
  ...
  # python3 interpreter
  python3 = {
    ...
    # helper function
    withPackages = ...
  };
  # adopted python3 packages
  python3Packages = {
    numpy = ...
    scipy = ...
    ...
  };
  ...
}