How to structure Flask applications part 2 - Growing up

My first introduction to Flask was Miguel Grinberg's original Flask Megatutorial. After using Django as my framework of choice for several years. Miguel's simple use of Flask allowed me to see just far enough under the hood that I started understanding much more of the web stack, hidden in some frameworks. Some consider Flask to bare bones. The transparency of the request/response cycle would help me understand how frameworks provide structure and functionality on top of a simple idea.

While building dozens of Flask applications, there was not much thought to the folder structure. As the applications evolve, refactors are necessary for developer sanity. While reviewing projects for this article, a pattern emerged.

I have been building poorly structured, poorly named, MVC frameworks.

Many frameworks use a Model/View/Controller architecture. The Conveyor application started from a boilerplate based on Miguel's tutorial. It has naturally evolved to its current state. Let's take a look:

conveyor
├── config.py
├── migrations/
├── requirements/
├── tests/
└── app/
    ├── wsgi.py
    ├── __init__.py
    ├── db.py
    ├── auth/
        ├── __init__.py
        ├── forms.py
        └── routes.py
    ├── server_providers/
        ├── __init__.py
        ├── routes.py
        ├── digitial_ocean/
        └── linode/
    ├── sites/
        ├── templates/
            ├── branch_select.html
            ├── create.html
            ├── deploy_button.html
            ├── deploy_script.html
            ├── index.html
            ├── page.html
            ├── push_to_deploy.html
            ├── repo_select.html
            └── show.html
        ├── details/
            ├── templates/
            ├── __init__.py
            ├── forms.py
            └── routes.py
        ├── events/
            ├── templates/
            ├── __init__.py
            └── routes.py
        ├── nginx/
            ├── templates/
            ├── __init__.py
            ├── forms.py
            └── routes.py
        ├── __init__.py
        ├── forms.py
        └── routes.py
    ├── templates/
        ├── auth/
        ├── git_providers/
        ├── server_providers/
        ├── base.html
        ├── footer.html
        ├── header.html
    ├── static/
        ├── images/
        ├── node_modules/
        ├── src/
        ├── package.json
        ├── styles.css
        └── webpack.config.js
    ├── __init__.py
    ├── db.py
    └── models.py

Overview of how this works:

• config.py - a file to hold our application configuration options for production, development, and testing environments. Common configuration options include SECRET_KEY and SQLALCHEMY_DATABASE_URI
• requirements.txt - a list of the project requirements
• test - a directory to house the tests for the application
• app - this directory holds our application code.
  • routes.py - contains your CRUD actions for pages and will include the logic for generating HTML or JSON requested by the client.
  • models.py - houses the data model for database objects
  • forms.py - a collection of forms used by the application. The app folder also contains a template folder for HTML templates.
  • app/__init__.py - The entry point for the application is __init__.py which initializes and configures the application. This also loads in each module blueprint.
  • module/__init__.py - these files are spread across the application to manage blueprints.

Blueprints

Concretely, the app level __init__.py registers blueprints in this fashion.

from app.auth import bp as auth_bp
app.register_blueprint(auth_bp, url_prefix='/auth')

from app.servers import bp as server_bp
app.register_blueprint(server_bp, url_prefix='/servers')

At the module level each __init__.py will create the blueprint and import the routes. Optionally, a template_folder is specified for modules that have their own templates directory.

from flask import Blueprint

bp = Blueprint('sites', __name__, template_folder="templates")
from app.sites import routes

How can this be improved?

• developer experience - As the project grows, some of the files are becoming very large. Changing between files in my workflow is cheap. The models and routes are prime for refactoring.
• models.py - this file is starting to get obnoxiously long. In the beginning, it made sense to keep the models in a single file. We are growing up now. A models folder will be added, and there will be individual files within each model. This method allows for from models import site instead of from site import models. A small difference, however, the difference is seen when you start importing many models. from models import server, server_provider, site instead of having to write different import line for each module.
• templates - Where should template files be stored? Currently, there is a template directory at the app level. While building Conveyor, I decided to try putting templates inside of each module instead of having them at the application level. The sites module is an excellent example of this - the sites/templates directory and each submodule also include a template folder. After trying this style out for a while it, I decided to use the application level app/templates structure moving forward.

The project structure

conveyor
├── config.py
├── migrations/
├── requirements/
├── tests/
└── app/
    ├── wsgi.py
    ├── __init__.py
    ├── db.py
    ├── models/
        ├── site.py
        ├── server.py
        ├── server_provider.py
    └── auth/
        ├── __init__.py
        ├── forms.py
        └── routes.py
    └── server_providers/
        ├── __init__.py
        ├── routes.py
        ├── digitial_ocean/
        └── linode/
    ├── sites/
        ├── details/
            ├── __init__.py
            ├── forms.py
            └── routes.py
        ├── events/
            ├── __init__.py
            └── routes.py
        ├── nginx/
            ├── __init__.py
            ├── forms.py
            └── routes.py
        ├── __init__.py
        ├── forms.py
        └── routes.py
    ├── static/
        ├── images/
        ├── node_modules/
        ├── src/
        ├── package.json
        ├── styles.css
        └── webpack.config.js
    ├── templates/
        ├── auth/
        ├── server_providers/
        ├── sites/
            ├── partials/
                ├── details/
                ├── events/
                ├── nginx/
                ├── branch_select.html
                ├── deploy_button.html
                ├── deploy_script.html
                ├── push_to_deploy.html
                ├── repo_select.html
            ├── create.html
            ├── index.html
            ├── page.html
            └── show.html
        ├── base.html
        ├── footer.html
        └── header.html

Whew, we did it

Making the changes above will give the application a slightly different structure and help keep it developer-friendly. There are some changes I would like to make to routes.py. Currently, the route files are doing a large amount of work. Handling everything from getting data, preparing data for the template, generating template HTML. The routes files also hold any additional logic that the views may need.

The idea here is to show you that your application structure can be flexible and can change as your application grows. Don't be afraid to try out your own structure and determine if it allows you to be productive.

If you have any feedback on how this can be improved further or have your way that you solve some of these problems reach out to me! mikeabrahamsen on Twitter