Implementation of Flask RESTful application with Docker and MongoDB


Introduction

The task is to create a simple RESTful Flask application that takes data from a database and a script that will fetch data from an external API and fills the database. All the code should be written on a Python and packed into a docker-compose (database + flask app).

 

Design

  • Database

As data, we use a pre-prepared selection of generated users using the randomuser service. The service provides an API with which you can get the desired number of users with predefined parameters or with default values. The result is returned in JSON, XML, CSV or YAML formats. We will work with the JSON format. 

MongoDB NoSQL database was used for data storage. The choice is due to the fact that we will work with JSON as a document, without creating additional tables and relationships between them. Therefore, there is no need for complex SQL queries.

  • Resources

Since the resource will display only user data, the root URL can be called:

http://[hostname]/api/users

According to the condition, the service displays and deletes user data, recording new and making changes is not required, therefore, we will use the following HTTP methods to access resource instances:

 

Метод HTTP     

URI

Действие

GET

http://[hostname]/api/users             

Получить список пользователей    

GET

http://[hostname]/api/users/{id}

Получить пользователя

DELETE

http://[hostname]/api/users/{id}

Удалить пользователя

 

Implementation

  • Environment

To run the application on localhost or on a remote server, we will use docker and docker-compose.  

Create the REST-API root directory with the docker-compose.yml file in which we will specify instructions for building the MongoDB image, server, and client:

version: '3.3'
services:  
  mongo:
    image: mongo:latest #download from Dockerhub latest ver
    container_name: mongo #name of running container
    ports: #ports that exposed to the host machine
     - "27017:27017"
  server:
    build: server/
    image: server
    container_name: server
    ports:
      - '5000:5000'
    volumes:
      - ./server:/server
    depends_on:
      - mongo # do not start before this container
    environment:
      - DEBUG=1 # 1/0
      - COUNT_USERS=100 # int
      - GENDER=male # male/female
  client:
    build: client/
    image: client
    container_name: client
    volumes:
      - ./client:/client
    ports:
      - '5001:5001'
    depends_on:
      - server # do not start before this container
    environment:
      - DEBUG=1

The version of the syntax is indicated in the file, in the services command we place the services and instructions on the basis of which the images will be created. 

 

  • Server

Create a server folder that will contain a script to populate the database, a Flask application for interacting with it, and the Dockerfile.

Create a file called fill_db.py which will contain the following code described below. To fill the database with users, we will use the pymongo library and requests. Also, we import the COUNT_USERS, GENDER variables from the config.py file, the value of which we can manipulate directly in the docker-compose.yml file.

import requests
from pymongo import MongoClient
from config import COUNT_USERS, GENDER

Next, we connect to the database, whose host is the name of the container, port 27017, which is the default for MongoDB, to expose from the container as indicated in the docker-compose file. 

client = MongoClient('mongo:27017')

We call the database for the MongoDB instance, for example, test_database:

db = client.test_database

Now we call the users collection on the database. In case the container is raised again, we clear the collection, this instruction may be deleted in case data must be saved.

users = db.users
users.remove({})

And after that, we declare the fill_db function in which we send a GET request with parameters (number of users and gender). 

def fill_db():
res = requests.get('https://randomuser.me/api/', params={'results': COUNT_USERS, 'gender': GENDER})
    try:
        users.insert_many(res.json()['results'])
    except Exception as e:
        raise e

We accept the JSON file, extract the list using the key ‘results’ and write it to the collection. In case of an error, raise an exception.

To send queries to the database, we need a server. Let's create the app.py file for creating and configuring the Flask application, import all the necessary modules, and a pre-prepared converter for converting the ObjectId element into a string data type for subsequent serialization and vice versa.

from flask import Flask, jsonify
from flask_pymongo import PyMongo
from pymongo import MongoClient
from converter import MongoJSONEncoder, ObjectIdConverter
from config import Configuration

app = Flask(__name__)
app.config.from_object(Configuration)
mongo = PyMongo(app)

# converting mongodb ObjectId format to string
app.json_encoder = MongoJSONEncoder
app.url_map.converters['objectid'] = ObjectIdConverter

In the config.py file, add a class to configure the application:

import os


COUNT_USERS = os.environ.get('COUNT_USERS')
GENDER = os.environ.get('GENDER')

class Configuration(object):
    DEBUG = os.environ.get('DEBUG')
    MONGO_URI = 'mongodb://mongo:27017/test_database'

The application is configured, the next step is to add routes for our resources and their processing. Since we will have one resource, we will use one instance of Blueprint. 

In the root directory, create the view.py file. We import the necessary modules and the client database.

from app import mongo
from flask import Blueprint, jsonify
from bson import ObjectId

We get the user collection and pass it to the variable:

collection = mongo.db.get_collection('users')

We create a Blueprint object which we will specify routes in the future.

users = Blueprint('users', __name__)

Let's define the routes. The route that will return the list of users:

# get list of users
@users.route('/api/users/', methods=['GET'])
def users_list():
    users = list(collection.find())
    if users:
        return jsonify(users)
    return jsonify({'message':'error', 'data':'db is empty'}), 404

When accessing the resource, all data from the collection is collected into a list; if the list is not empty, the server returns a JSON file with a list of users. Otherwise, it causes a 404 error and transmits a JSON file with the status and message.

The route that will return the user by id. At first, we must check user_id which came from URL, if it is an instance of ObjectId. If true, we search him at the database. If not, return the JSON file with an error message and 404 status. When the user presents function return JSON file with user data. Else return an error message and 404 status.

# get single user by id
@users.route('/api/users/<user_id>', methods=['GET'])
def get_user(user_id):
    if ObjectId.is_valid(user_id):
    user = collection.find_one({'_id':ObjectId(user_id)}
    if user:
        return jsonify(user)
    return jsonify({'message':'error', 'data':'user not found'}), 404

  return jsonify({'message':'error', 'data':'invalid user id'}), 404

The route that will delete the user:

# delete single user by id
@users.route('/api/users/<user_id>', methods=['DELETE'])
def delete_user(user_id):
    if ObjectId.is_valid(user_id):
        if collection.find_one({'_id':ObjectId(user_id)}):
            collection.delete_one({'_id':ObjectId(user_id)})
            return jsonify({'message':'ok', 'data':{'status':'deleted', 'user':user_id}})
        return jsonify({'message':'error', 'data':'user not found'}), 404
    return jsonify({'message':'error', 'data':'invalid user id'}), 404

Routes are ready, left to add an entry point, list of dependencies, and Dockerfile.

Create the main.py file from which our application will be launched. 

from app import app
from view import users
from fill_db import fill_db

app.register_blueprint(users)

if __name__=="__main__":
    fill_db()
    app.run(host='0.0.0.0', port=5000)

Create the requirements.txt file and place the dependencies in it:

click==7.1.1
Flask==1.1.2
Flask-PyMongo==2.3.0
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
pymongo==3.10.1
Werkzeug==1.0.1
gunicorn==20.0.4
certifi==2020.4.5.1
chardet==3.0.4
idna==2.9
pymongo==3.10.1
requests==2.23.0
urllib3==1.25.9

Dockerfile contains instructions for building the image of our application. Download the python image version 3.7. We create the server folder inside the container, make it work for the following instructions. We copy the local folder to the server folder that we created two commands above. Run the commands for updating pip and installing dependencies, as well as the nano code editor. ENV instruction allows you to specify variables and their values. They will be available inside the container as environment variables. Run the script to populate the database. We launch the application.

FROM python:3.7
RUN mkdir -p /server/
WORKDIR /server/
COPY . /server/
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt
RUN apt-get update
RUN apt-get install nano #for debugging
ENV DEBUG=1
ENV COUNT_USERS=100
ENV GENDER=female
ENTRYPOINT ["python", "fill_db.py"] #fill database
ENTRYPOINT ["python", "main.py"] #run flask app
EXPOSE 5000

Since the condition of the task was not to organize a check of access to the service, we do not touch on the topic of security and authorization.

Microservice

To visualize the data, a microservice was implemented on Flask with a minimal design of HTML pages from Bootstrap templates.

The microservice was also configured in a separate container and interacts with the server using the HTTP protocol.

Application launch

To launch the application, go to the root folder of the application and use the command:

    docker-compose up --build

Access to the API can be obtained by going to the address: http://localhost:5000/api/users/

Visualization microservice is available at http://localhost:5001/users/

For testing API resources, the Postman service was used.

. . .


Comments 0:


Login for commenting