Skip to content

Getting started with FlowKit

Installation requirements

Most FlowKit components (FlowDB, FlowMachine, FlowAPI, FlowAuth, FlowETL, AutoFlow) are distributed as docker containers. To install these, you need:

  • docker >= 17.12.0
  • docker-compose >= 1.21.0

In addition, running FlowClient requires:

  • Python >= 3.6

There are additional requirements for a development setup, and we strongly recommend reviewing the administrator section in detail if you are planning to deploy FlowKit in production.

Quick install

This quick install guide will install the major components of FlowKit together with an initial setup and example analysis query.

The bulk of the installation process consists of using Docker Compose to download Docker containers from Docker Hub, followed by a pip install of FlowClient.

These instructions assume use of Pyenv and Pipenv. If you are using Anaconda-based installation commands may be different.

Docker containers for FlowAPI, FlowMachine, FlowDB, FlowAuth and the worked examples are provided in the Docker Hub repositories flowminder/flowapi, flowminder/flowmachine, flowminder/flowdb, flowminder/flowauth, and flowminder/flowkit-examples respectively. To install them, you will need Docker and Docker Compose.

Ensure that you have a Docker installed and running, then start the FlowKit test system by running

bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh)

This will pull any necessary docker containers, and start the system in the background with the API exposed on port 9090 by default, and the FlowAuth authentication system accessible by visiting http://localhost:9091 using your web browser.

The default system includes a small amount of test data. For a test system with considerably more data you can run

bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) larger_data

Warning

The larger data container will take considerably longer to start up, as it generates data when first run.

The worked examples are also available as part of the demo system. To install these run

bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) examples smaller_data

for the examples with a small dataset, or

bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) examples

to get the examples with the larger dataset (the one used when producing this documentation).

You can also explore FlowKit using synthetic data for other countries, by setting the EXAMPLE_COUNTRY environment variable to a three-level ISO country code, for example to simulate data for the UK with a crisis in Greater London:

EXAMPLE_COUNTRY=GBR EXAMPLE_DISASTER_REGION_PCOD=GBR.1.36_1 bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) examples

Available countries are those with boundaries in the GADM 3.6 dataset. Possible values for EXAMPLE_DISASTER_REGION_PCOD are any admin2 region for the corresponding country, based on the GID_2 field in the GADM 3.6 shapefile for that country.

Info

The small dataset is sufficient for most of the worked examples, but the larger dataset is required for the Flows Above Normal example because this uses data for dates outside the range included in the small dataset.

Info

The worked examples make use of Mapbox GL for visualisation, which requires an API access token. If you would like to produce the maps in the worked examples notebooks, you will need to create a mapbox access token (following instructions here), and set this as the value of the MAPBOX_ACCESS_TOKEN environment variable before running the above commands.

To shut down the system, you can either stop all the docker containers directly, or run

bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) stop

In order to use the test system, now install FlowClient, and generate a token using FlowAuth.

FlowAuth quickstart

Visit http://localhost:9091 and log in with either TEST_ADMIN:DUMMY_PASSWORD or TEST_USER:DUMMY_PASSWORD. TEST_USER is already set up to generate tokens for the FlowAPI instance started by the quick start script.

See the administrator section for details of how to add servers and users or modify user permissions, or the analyst section for instructions to generate a token.

FlowClient

The FlowClient Python client is used to perform CDR analysis using the JupyterLab Python Data Science Stack. It may be installed using pip:

pip install flowclient

Quick install is continued with an example of FlowClient usage here.

If you are planning to hack on the codebase, you should also check out the developer install instructions; if you are ready to try FlowKit in production, visit the production deployment documentation to find out how to deploy FlowKit at your site.