Upload Box Open on Alt Tab in Flask

What is Flask-Static-Digest?

It is a Flask extension that will aid make your static files product ready with very minimal endeavour on your function. Information technology does this by md5 tagging and gzipping your static files after running a flask digest compile command that this extension adds to your Flask app.

It should be the last thing you do to your static files before uploading them to your server or CDN. Speaking of which, if you're using a CDN this extension optionally lets y'all configure a host URL that will become prepended to your static file paths. If you're non using a CDN, no problem everything will piece of work as you lot would expect by default.

Other web frameworks like Django, Carmine on Rails and Phoenix all have this feature congenital into their framework, and at present with this extension Flask does too.

This extension will piece of work if you lot're not using any asset build tools just at the same fourth dimension information technology as well works with Webpack, Grunt, Gulp or any other build tool y'all tin think of. This tool does not depend on or compete with existing asset build tools.

If yous're already using Webpack or a similar tool, that's great. Webpack takes care of bundling your assets and helps convert things like SASS to CSS and ES6+ JS to browser compatible JS. That is solving a completely different problem than what this extension solves. This extension volition further optimize your static files after your build tool produces its output files.

This extension does things that Webpack solitary cannot do considering in order for things like md5 tagging to work Flask needs to be aware of how to map those hashed file names dorsum to regular file names you would reference in your Jinja 2 templates.

How does it piece of work?

There's iii pieces to this extension:

Information technology adds a custom Flask CLI command to your project. When you run this command information technology looks at your static files and so generates an md5 tagged version of each file forth with optionally gzipping them as well.
When the in a higher place command finishes information technology creates a cache_manifest.json file in your static folder which maps the regular file names, such as images/flask.png to images/flask-f86b271a51b3cfad5faa9299dacd987f.png.
Information technology adds a new template helper chosen static_url_for which uses Flask's url_for under the hood but is aware of the cache_manifest.json file and so information technology knows how to resolve images/flask.png to the md5 tagged file name.

Demo video

This 25 infinitesimal video goes over using this extension but information technology as well spends a lot of time on the "why" where we embrace topics like cache busting and why IMO you might want to use this extension in all of your Flask projects.

If y'all adopt reading instead of video, this README file covers installing, configuring and using this extension too.

Demo Video

Changes since this video

FLASK_STATIC_DIGEST_HOST_URL has been added to configure an optional external host, aka. CDN (explained here)

Installation
Using the newly added Flask CLI control
Going over the Flask CLI commands
Configuring this extension
Modifying your templates to use static_url_for instead of url_for
Potentially updating your .gitignore file
FAQ
- What most development vs production and operation implications?
- Why bother gzipping your static files hither instead of with nginx?
- How do you use this extension with Webpack or another build tool?
- Migrating from Flask-Webpack
- How do yous use this extension with Docker?
- How practise y'all use this extension with Heroku?
- What about user uploaded files?
About the writer

Installation

You'll need to be running Python 3.5+ and using Flask 1.0 or greater.

pip install Flask-Static-Digest

Example directory structure for a 'hello' app

              ├── hello │   ├── __init__.py │   ├── app.py │   └── static │       └── css │           ├── app.css └── requirements.txt

Flask app mill example using this extension

              from              flask              import              Flask              from              flask_static_digest              import              FlaskStaticDigest              flask_static_digest              =              FlaskStaticDigest()              def              create_app():              app              =              Flask(__name__)              flask_static_digest.init_app(app)              @app.route("/")              def              index():              return              "Hello, World!"              return              app

A more than complete example app can be institute in the tests/ directory.

Using the newly added Flask CLI command

You'll desire to brand sure to at least set the FLASK_APP environment variable:

              consign              FLASK_APP=hello.app              export              FLASK_ENV=evolution

So run the flask binary to run into its aid menu:

Usage: flask [OPTIONS] Control [ARGS]...    ...  Options:   --version  Show the flask version   --help     Show this message and get out.  Commands:   digest  md5 tag and gzip static files.   routes  Evidence the routes              for              the app.   run     Run a development server.   shell   Run a shell              in              the app context.

If all went every bit planned you lot should see the new digest command added to the list of commands.

Going over the Flask CLI commands

Running flask digest will produce this assist menu:

Usage: flask digest [OPTIONS] Command [ARGS]...    md5 tag and gzip static files.  Options:   --help  Bear witness this message and exit.  Commands:   clean    Remove generated static files and cache manifest.   compile  Generate optimized static files and a cache manifest.

Each control is labeled, but here's a bit more data on what they practice.

compile

Inspects your Flask app's static_folder and uses that every bit both the input and output path of where to wait for and create the newly digested and compressed files.

At a high level information technology recursively loops over all of the files it finds in that directory and and then generates the md5 tagged and gzipped versions of each file. Information technology also creates a cache_manifest.json file in the root of your static_folder.

That manifest file is motorcar generated meaning you lot should not edit it unless you really know what you're doing.

This file maps the human readable file name of permit'south say images/flask.png to the digested file proper name. It's a elementary key / value set up. Information technology'southward basically a Python lexicon in JSON format.

In the cease information technology ways if your static folder looked similar this originally:

css/app.css
js/app.js
images/flask.png

And y'all decided to run the compile command, information technology would at present await like this:

css/app.css
css/app.css.gz
css/app-5d41402abc4b2a76b9719d911017c592.css
css/app-5d41402abc4b2a76b9719d911017c592.css.gz
js/app.js
js/app.js.gz
js/app-098f6bcd4621d373cade4e832627b4f6.js
js/app-098f6bcd4621d373cade4e832627b4f6.js.gz
images/flask.png
images/flask.png.gz
images/flask-f86b271a51b3cfad5faa9299dacd987f.png
images/flask-f86b271a51b3cfad5faa9299dacd987f.png.gz
cache_manifest.json

Your md5 hashes will exist different because it depends on what the contents of the file are.

clean

Inspects your Flask app's static_folder and uses that as the input path of where to look for digested and compressed files.

It will recursively delete files that have a file extension of .gz and also deletes files that have been digested. It determines if a file has been digested based on its file proper noun. In other words, it will delete files that match this regexp r"-[a-f\d]{32}".

In the end that means if you had these iv files in your static folder:

images/flask.png
images/flask.png.gz
images/flask-f86b271a51b3cfad5faa9299dacd987f.png
images/flask-f86b271a51b3cfad5faa9299dacd987f.png.gz

And yous decided to run the clean command, the last 3 files would be deleted leaving you with the original images/flask.png.

Configuring this extension

By default this extension volition md5 tag all files it finds in your configured static_folder. Information technology volition also create gzipped versions of each file and information technology won't prefix your static files with an external host.

If you don't like whatsoever of this behavior, you tin optionally configure:

              FLASK_STATIC_DIGEST_BLACKLIST_FILTER              =              []              # If you desire specific extensions to non go md5 tagged you can add them to              # the listing, such as: [".htm", ".html", ".txt"]. Brand sure to include the ".".              FLASK_STATIC_DIGEST_GZIP_FILES              =              True              # When prepare to False then gzipped files will not be created only static files              # will yet get md5 tagged.              FLASK_STATIC_DIGEST_HOST_URL              =              None              # When set to a value such every bit https://cdn.instance.com and you utilize static_url_for              # it will prefix your static path with this URL. This would be useful if you              # host your files from a CDN. Make sure to include the protocol (aka. https://).

You can override these defaults in your Flask app's config file.

Modifying your templates to use static_url_for instead of url_for

We're all familiar with this code right?

              <              img              src="{{ url_for('static', filename='images/flask.png') }}"              width="480"              height="188"              alt="Flask logo" />

When you put the above code into a Flask powered Jinja two template, information technology turns into this:

              <              img              src="images/flask.png"              width="480"              height="188"              alt="Flask logo" />

The path might vary depending on how you configured your Flask app's static_folder but you get the idea.

Using static_url_for instead of url_for

Let'southward use the same example as above:

              <              img              src="{{ static_url_for('static', filename='images/flask.png') }}"              width="480"              meridian="188"              alt="Flask logo" />

But now have a look at the output this produces:

              <              img              src="/images/flask-f86b271a51b3cfad5faa9299dacd987f.png"              width="480"              pinnacle="188"              alt="Flask logo" />

Or if you lot set FLASK_STATIC_DIGEST_HOST_URL = "https://cdn.example.com" it would produce:

              <              img              src="https://cdn.case.com/images/flask-f86b271a51b3cfad5faa9299dacd987f.png"              width="480"              height="188"              alt="Flask logo" />

Instead of using url_for you would utilise static_url_for. This uses Flask's url_for nether the hood and so things like _external=Truthful and everything else url_for supports is bachelor to apply with static_url_for.

That ways to use this extension you lot don't have to do anything other than install information technology, optionally run the CLI control to generate the manifest and then rename your static file references to use static_url_for instead of url_for.

If your editor supports performing a find / supervene upon across multiple files y'all can rapidly make the change by finding url_for('static' and replacing that with static_url_for('static'. If you happen to employ double quotes instead of single quotes you'll want to suit for that also.

Potentially updating your .gitignore file

If you're using something similar Webpack then chances are you're already git ignoring the static files it produces equally output. It's a common design to commit your Webpack source static files simply ignore the compiled static files it produces.

Only if you're non using Webpack or another asset build tool then the static files that are a part of your projection might take the same source and destination directory. If that's the case, chances are y'all'll want to git ignore the md5 tagged files as well as the gzipped and cache_manifest.json files from version command.

For clarity, you lot want to ignore them because you'll exist generating them on your server at deploy time or within a Docker prototype if you're using Docker. They don't need to be tracked in version control.

Add this to your .gitignore file to ignore certain files this extension creates:

              *-[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f].* *.gz cache_manifest.json

This allows your original static files merely ignores everything else this extension creates. I am enlightened at how ridiculous that ignore dominion is for the md5 hash but using [0-9a-f]{32} does not work. If you know of a better manner, please open up a PR!

FAQ

What about development vs production and performance implications?

Y'all would typically only run the CLI command to set your static files for production. Running flask digest compile would go a part of your build process -- typically after you pip install your dependencies.

In evolution when the cache_manifest.json probable doesn't exist static_url_for calls url_for directly. This allows the static_url_for helper to work in both evolution and product without whatsoever fuss.

It'south besides worth pointing out the CLI command is expected to exist run before you even start your Flask server (or gunicorn / etc.), so there's no perceivable run time performance hit. It simply involves doing one actress dictionary lookup at run time which is many orders of magnitude faster than fifty-fifty the about uncomplicated database query.

In other words, this extension is not going to negatively bear upon the performance of your spider web application. If annihilation it's going to speed information technology up and save yous money on hosting.

That'south because gzipped files can exist upwardly of 5-10x smaller so in that location'southward less bytes to transfer over the network.

Also with md5 tagging each file it means you can configure your web server such as nginx to cache each file forever. That ways if a user visits your site a second time in the time to come, nginx will be smart enough to load it from their local browser'due south cache without even contacting your server. It'southward a 100% local look up.

This is equally efficient equally it gets. You lot tin't do this normally without md5 tagging each file because if the file changes in the time to come, nginx will go on serving the old file until the cache expires so users will never see your updates. Simply due to how md5 hashing works, if the contents of a file changes information technology volition get generated with a new proper noun and nginx will serve the uncached new file.

This tactic is commonly referred to as "enshroud busting" and it's a very skillful thought to do this in production. You can fifty-fifty go 1 stride further and serve your static files using a CDN. Using this cache busting strategy makes configuring your CDN a easy since you don't need to worry about e'er expiring your cache manually.

Why bother gzipping your static files here instead of with nginx?

You would still be using nginx's gzip features, but at present instead of nginx having to gzip your files on the fly at run fourth dimension you can configure nginx to employ the pre-made gzipped files that this extension creates.

This way you tin can benefit from having maximum compression without having nginx waste precious CPU cycles gzipping files on the fly. This gives you lot the best of both worlds -- the highest compression ratio with no noticeable run time performance penalty.

How do you use this extension with Webpack or some other build tool?

It works out of the box with no extra configuration or plugins needed for Webpack or your build tool of choice.

Typically the Webpack (or another build tool) work menses would look like this:

You configure Webpack with your source static files directory
You configure Webpack with your destination static files directory
Webpack processes your files in the source directory and copies them to the destination directory
Flask is configured to serve static files from that destination directory

For example, your source directory might be avails/ inside of your project and the destination might be myapp/static.

This extension will expect at your Flask configuration for the static_folder and determine it's prepare to myapp/static so it will md5 tag and gzip those files. Your Webpack source files will not go digested and compressed.

Migrating from Flask-Webpack

Flask-Webpack is another extension I wrote a long time ago which was specific to Webpack simply had a similar idea to this extension. Flask-Webpack is now deprecated in favor of Flask-Static-Digest. Migrating is fairly painless. In that location are a number of changes but on the bright side y'all become to delete more code than you lot add together!

Dependency / Flask app changes

Remove Flask-Webpack from requirements.txt
Remove all references to Flask-Webpack from your Flask app and config
Remove manifest-revision-webpack-plugin from package.json
Remove all references to this webpack plugin from your webpack config
Add Flask-Static-Digest to requirements.txt
Add the Flask-Static-Digest extension to your Flask app

Jinja 2 template changes

Supplant stylesheet_tag('main_css') | safety with static_url_for('static', filename='css/main.css')
Supervene upon javascript_tag('main_js') | rubber with static_url_for('static', filename='js/principal.js')
Replace whatsoever occurrences of asset_url_for('foo.png') with static_url_for('static', filename='images/foo.png')

How do y'all use this extension with Docker?

It's really no different than without Docker, simply instead of running flask assimilate compile on your server direct at deploy fourth dimension you would run it inside of your Docker image at build time. This style your static files are already set up up and ready to go by the fourth dimension you pull and use your Docker image in product.

Yous can see a fully working instance of this in the open up source version of my Build a SAAS App with Flask course. It leverages Docker's build arguments to only compile the static files when FLASK_ENV is ready to production. The key files to look at are the Dockerfile, docker-compose.yml and .env files. That wires up the build arguments and env variables to make it work.

How do you utilise this extension with Heroku?

If you're deploying to Heroku using the Python buildpack y'all tin can follow these 2 steps:

Create a bin/post_compile file in your projection's source code
Re-create the lines below into the bin/post_compile file, save it and commit the changes

                              #!/usr/bin/env bash              set              -e              echo                              "-----> Digesting static files"                            cd                              "                ${1}                "                            &&              flask digest compile

The next time y'all push your code this script volition run later your pip dependencies are installed. Information technology will run before your slug is compiled which ensures that the digested files are available earlier any traffic is served to your Dyno.

You can view how this file gets executed by Heroku in their Python buildpack's source lawmaking.

What about user uploaded files?

Let'south say that besides having static files like your logo and CSS / JavaScript bundles you besides accept files uploaded by users. This could be things like a user avatar, blog post images or anything else.

You would still want to md5 tag and gzip these files merely now we've run into a situation. The flask assimilate compile command is meant to be run at deploy fourth dimension and it could potentially be run from your dev box, inside of a Docker image, on a CI server or your production server. In these cases you lot wouldn't have access to the user uploaded files.

But at the same time you have users uploading files at run time. They are irresolute all the time.

Needless to say you can't utilize the flask digest compile control to digest user uploaded files. The cache_manifest.json file should be reserved for files that exist in your lawmaking repo (such as your CSS / JS bundles, maybe a logo, fonts, etc.).

The above files do not change at run fourth dimension and align well with running the flask digest compile command at deploy time.

For user uploaded content you lot wouldn't always write these entries to the manifest JSON file. Instead, you lot would typically upload your files to disk, S3 or somewhere else and then salve the file name of the file you uploaded into your local database.

And so at present when you lot reference a user uploaded file (let'south say an avatar), you would loop over your users from the database and reference the file name from the DB.

In that location's no need for a manifest file to store the user uploaded files because the database has a reference to the existent proper name and so you are dynamically referencing that in your template helper (static_url_for), so it's never a hard coded matter that changes at the template level.

What's cool about this is you already did the database query to call back the record(s) from the database, so there'south no actress database work to do. All you take to do is reference the file name field that's a part of your model.

But that doesn't fully solve the problem. You'll still desire to md5 tag and gzip your user uploaded content at run fourth dimension and you would want to exercise this before you save the uploaded file into its final destination (local file organisation, S3, etc.).

This tin can be washed completely divide from this extension and it's really going to vary depending on where y'all host your user uploaded content. For instance some CDNs will automatically create gzipped files for you and they use things like an ETag header in the response to include a unique file name (and this is what you can shop in your DB).

So perchance md5 hashing and maybe gzipping your user uploaded content becomes an app specific responsibility, although I'yard not opposed to maybe creating helper functions you lot can use just that would need to be idea out advisedly.

However the implementation is not bad. It's actually only about five lines of code to do both things. Feel free to CTRL + F around the lawmaking base for hashlib and gzip and you'll find the related code.

So with that said, here's a work flow you can practise to deal with this today:

User uploads file
Your Flask app potentially md5 tags / gzips the file if necessary
Your Flask app saves the file proper name + gzipped file to its final destination (local file system, S3, etc.)
Your Flask app saves the final unique file proper noun to your database

That concluding unique file name would be the md5 tagged version of the file that yous created or the unique file proper name that your CDN returned back to you. I hope that clears up how to deal with user uploaded files and efficiently serving them!

About the author

Nick Janetakis | https://nickjanetakis.com | @nickjanetakis

If yous're interested in learning Flask I have a 17+ 60 minutes video course called Build a SAAS App with Flask. It's a form where nosotros build a real globe SAAS app. Everything about the course and demo videos of what nosotros build is on the site linked to a higher place.