Implementation of Flask RESTful application with Docker


Introduction

The task is to create a simple RESTful Flask application that takes data from a database and a script that will fetch data from an external API and fills the database. All the code should be written on a Python and packed into a docker-compose (database + flask app).

 

Design

  • Database

As data, we use a pre-prepared selection of generated users using the randomuser service. The service provides an API with which you can get the desired number of users with predefined parameters or with default values. The result is returned in JSON, XML, CSV or YAML formats. We will work with json format. 

MongoDB nosql database was used for data storage. The choice is due to the fact that we will work with json as a document, without creating additional tables and relationships between them. Therefore, there is no need for complex SQL queries.

  • Resources

Since the resource will display only user data, the root URL can be called:

http://[hostname]/api/users

According to the condition, the service displays and deletes user data, recording new and making changes is not required, therefore, we will use the following HTTP methods to access resource instances:

 

Метод HTTP     

URI

Действие

GET

http://[hostname]/api/users             

Получить список пользователей    

GET

http://[hostname]/api/users/{id}

Получить пользователя

DELETE

http://[hostname]/api/users/{id}

Удалить пользователя

 

Implementation

  • Environment

To run the application on a local host or on a remote server, we will use docker and docker-compose.  

Create the REST-API root directory with the docker-compose.yml file in which we will specify instructions for building the MongoDB image, server and client:

 

 

version: '3.3'

services:  

  mongo:

    image: mongo:latest #download from Dockerhub latest ver

    container_name: mongo #name of running container

    ports: #ports that exposed to the host machine

     - "27017:27017"

  server:

    build: server/

    image: server

    container_name: server

    ports:

      - '5000:5000'

    volumes:

      - ./server:/server

    depends_on:

      - mongo # do not start before this container

    environment:

      - DEBUG=1 # 1/0

      - COUNT_USERS=100 # int

      - GENDER=male # male/female

  client:

    build: client/

    image: client

    container_name: client

    volumes:

      - ./client:/client

    ports:

      - '5001:5001'

    depends_on:

      - server # do not start before this container

    environment:

      - DEBUG=1

 

The version of the syntax is indicated in the file, in the services command we place the services and instructions on the basis of which the images will be created. 

 

  • Server

Create a server folder that will contain a script to populate the database, a Flask application for interacting with it and the Dockerfile.

Create a file called fill_db.py which will contain the following code described below. To fill the database with users, we will use the pymongo library and requests. Also, we import the COUNT_USERS, GENDER variables from the config.py file, the value of which we can manipulate directly in the docker-compose.yml file.

import requests
from pymongo import MongoClient
from config import COUNT_USERS, GENDER

Next, we connect to the database, whose host is the name of the container, port 27017, which is the default for mongodb, to expose from the container as indicated in the docker-compose file. 

 

client = MongoClient('mongo:27017')

 

We call the database for the mongodb instance, for example test_database:

 

db = client.test_database

 

Now we call the users collection on the database. In case the container is raised again, we clear the collection, this instruction may be deleted in case data must be saved.

 

users = db.users
users.remove({})

 

 

And after that we declare the fill_db function in which we send a GET request with parameters (number of users and gender). 

 

def fill_db():
res = requests.get('https://randomuser.me/api/', params={'results': COUNT_USERS, 'gender': GENDER})
    try:
        users.insert_many(res.json()['results'])
    except Exception as e:
        raise e

 

We accept the json file, extract the list using the key ‘results’ and write it to the collection. In case of an error, raise an exception.

To send queries to the database, we need a server. Let's create the app.py file for creating and configuring the Flask application, import all the necessary modules and a pre-prepared converter for converting the ObjectId element into a string data type for subsequent serialization and vice versa.

 

from flask import Flask, jsonify
from flask_pymongo import PyMongo
from pymongo import MongoClient
from converter import MongoJSONEncoder, ObjectIdConverter
from config import Configuration

app = Flask(__name__)
app.config.from_object(Configuration)
mongo = PyMongo(app)

# converting mongodb ObjectId format to string
app.json_encoder = MongoJSONEncoder
app.url_map.converters['objectid'] = ObjectIdConverter

 

In the config.py file, add a class to configure the application:

 

import os

COUNT_USERS = os.environ.get('COUNT_USERS')

GENDER = os.environ.get('GENDER')

class Configuration(object):

    DEBUG = os.environ.get('DEBUG')

    MONGO_URI = 'mongodb://mongo:27017/test_database'

 

The application is configured, the next step is to add routes for our resources and their processing. Since we will have one resource, we will use one instance of Blueprint. 

In the root directory, create the view.py file. We import the necessary modules and the client database.

 

from app import mongo

from flask import Blueprint, jsonify

from bson import ObjectId

 

We get the user collection and pass it to the variable:

 

collection = mongo.db.get_collection('users')

 

We create a Blueprint object which we will specify routes in the future.

 

users = Blueprint('users', __name__)

 

Now define the routes.

The route that will return the list of users:

 

# get list of users

@users.route('/api/users/', methods=['GET'])

def users_list():

    users = list(collection.find())

    if users:

         return jsonify(users)

    return jsonify({'message':'error', 'data':'db is empty'}), 404

 

When accessing the resource, all data from the collection is collected into a list; if the list is not empty, the server returns a json file with a list of users. Otherwise, it causes a 404 error and transmits a json file with the status and message.

The route that will return the user by id. At first we must check user_id wich came from url, if it isinstance of ObjectId. If true, we search him at database. If not, return json file with error message and 404 status. When user present function return json file with user data. Else return error message and 404 status.

# get single user by id

@users.route('/api/users/<user_id>', methods=['GET'])

def get_user(user_id):

    if ObjectId.is_valid(user_id): 

        user = collection.find_one({'_id':ObjectId(user_id)})

        if user:

            return jsonify(user)

        return jsonify({'message':'error', 'data':'user not found'}), 404

    return jsonify({'message':'error', 'data':'invalid user id'}), 404

 

The route that will delete the user:

 

# delete single user by id

@users.route('/api/users/<user_id>', methods=['DELETE'])

def delete_user(user_id):

    if ObjectId.is_valid(user_id):

         if collection.find_one({'_id':ObjectId(user_id)}):

              collection.delete_one({'_id':ObjectId(user_id)})

              return jsonify({'message':'ok', 'data':{'status':'deleted',

                  'user':user_id}})

        return jsonify({'message':'error', 'data':'user not found'}), 404

    return jsonify({'message':'error', 'data':'invalid user id'}), 404

 

 

Routes are ready, left to add an entry point, list of dependencies and Dockerfile.

Create the main.py file from which our application will be launched. 

 

from app import app

from view import users

from fill_db import fill_db

 

app.register_blueprint(users)

 

if __name__=="__main__":

    fill_db()

    app.run(host='0.0.0.0', port=5000)

 

Create the requirements.txt file and place the dependencies in it:

 

click==7.1.1

Flask==1.1.2

Flask-PyMongo==2.3.0

itsdangerous==1.1.0

Jinja2==2.11.2

MarkupSafe==1.1.1

pymongo==3.10.1

Werkzeug==1.0.1

gunicorn==20.0.4

certifi==2020.4.5.1

chardet==3.0.4

idna==2.9

pymongo==3.10.1

requests==2.23.0

urllib3==1.25.9

 

Dockerfile contains instructions for building the image of our application. Download the python image version 3.7. We create the server folder inside the container, make it working for the following instructions. We copy the local folder to the server folder that we created two commands above. Run the commands for updating pip and installing dependencies, as well as the nano code editor. The ENV instruction allows you to specify variables and their values. They will be available inside the container as environment variables. Run the script to populate the database. We launch the application.

 

FROM python:3.7

RUN mkdir -p /server/

WORKDIR /server/

COPY . /server/

RUN pip install --upgrade pip

RUN pip install --no-cache-dir -r requirements.txt

RUN apt-get update

RUN apt-get install nano #for debugging

ENV DEBUG=1

ENV COUNT_USERS=100

ENV GENDER=female

ENTRYPOINT ["python", "fill_db.py"] #fill database

ENTRYPOINT ["python", "main.py"] #run flask app

EXPOSE 5000

 

Since the condition of the task was not to organize a check of access to the service, we do not touch on the topic of security and authorization.

 

Microservice

 

To visualize the data, a microservice was implemented on Flask with a minimal design of html pages from Bootstrap templates.

The microservice was also configured in a separate container and interacts with the server using the HTTP protocol.

 

Application launch

 

To launch the application, go to the root folder of the application and use the command:

    docker-compose up --build

Access to the API can be obtained by going to the address: http://localhost:5000/api/users/

Visualization microservice is available at: http://localhost:5001/users/

For testing API resources, the Postman service was used.

 

. . .


Comments 0:


Login for commenting