Django Migrations and Database Connectivity – An Excellent concept made Easy!

Free Python courses with 57 real-time projects - Learn Python

FREE Online Courses: Knowledge Awaits – Click for Free Access!

Django is an awesome framework. It abstracts the most trivial tasks and allows for rapid development. Since Django has an active community, they are constantly adding new features to it. These features give Django an edge in web development.

One such feature was added with the introduction of Django version 1.7, that is, Migrations which was pretty much required everywhere. It is important for defining the database schema. We also use them to maintain complex databases whenever changes are performed in them.

We will learn about Django migrations and various other concepts related to it.

Django Migrations and Database Connectivity

Awesome!! We have so much information to grasp. So, let’s explore it.

Databases and Django

A database is a collection of data arranged in an efficient way. It ensures fast and secure storage, access and update of data. That is, the main concept of every type of database is both paper registers and MySQL.

There are many types of databases out there. We will be narrowing them down to relational databases. Now, what’s that? A relational database is a collection of tables having a number of columns and rows.

Yes, there are databases that actually don’t use columns and rows. So, whatever databases we have worked with on the web is a relational database. These databases can have relational tables or fields. We have used Foreign Key Field in our Django projects.

Here, we are creating a post involving a Foreign Key for Post Table.

Foreign key for Post table

This is a snapshot of the models.py file of our project.

SQL (Structured Query Language) is a very popular language for databases. Almost all the popular databases use SQL. The databases like PostgreSQL, MySQL, SQLite, etc. are some of many examples.

The databases mentioned above are relational databases. Yes, the majority of SQL databases are relational and are extensively used in the industry.

What’s special about PostgreSQL, MySQL, etc. is that they have native support from Django. SQLite3 is the one that comes installed out of the box, although, you will have to install the software for other databases. Django ORM will handle the rest according to the backend engine used.

We will discuss the configuration of these databases in a separate article. It will provide you with all the required information and Django settings. Now, let’s learn some important configuration of settings for databases.

Configuring Settings for Django Database

These are common settings you should know to connect database with Django project. This code exists in the settings.py file.

Common Settings to Connect Database with Django project

Here is the snapshot of the default code that Django provides.

Basically, database settings for your project reside in a Python Dictionary. The name of the dictionary is DATABASE. This dictionary and some of its attributes are pre-defined.

The default key is pre-defined, which contains configuration settings for a default database. Since Django can work with multiple databases, you can have multiple database configurations.

The default key is required to configure a single or multiple databases. Inside the default value, key resides the configuration for the database.

The configuration can vary for different types of the database you are using.

The default value is:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'mydatabase',
    }
}

Now, these keys have different values for different databases. This configuration is for SQLite database.

The ENGINE Key is used to specify the database backend. This will connect the Django project to SQLite Database. You can use other backends too. For that, you have to provide additional keys. We will configure these databases in a separate article.

Note:

Django comes with these four database backends natively. These databases will require their database software but the backend is in-house.

PostgreSQL:

django.db.backends.postgresql

MySQL:

django.db.backends.mysql

SQLite:

django.db.backends.sqlite3

Oracle:

django.db.backends.oracle

These backends will be able to implement all the latest features of Django. They are developed and maintained by the DSF (Django Software Foundation). You can use other backends as well. In that case, you have to import the full package path of that backend.

Since Django has a vast community, you can get backends for all kinds of databases.

The NAME Key contains the name of the database. For SQLite, you can directly use this database. For databases like MySQL or PostgreSQL, you need to create the database first.

So, these were two important fields for configuring databases in Django. You can complete our Django Database DataFlair tutorial to implement the code in this tutorial.

How these Databases Work with Django

In this section, we are going to understand how these databases work with Django. After you have configured your database settings and checked that everything is working fine, see what happens under the hood.

When we start our Django project, it loads the settings.py file on the server. This file contains all information and even database configurations.

Then Django will load the Database Backend. That backend will connect to the default database configured in settings. It is done by passing appropriate key values to the Database Software.

The database then verifies the values and the connection is established. The connection is between the Database Backend and Database. The backend becomes a mediator between Database and Django.

All the built-in backends are very efficient and are used in a production environment.

Now, the database is connected and we can easily migrate our models and create tables in the same. The backend also handles all the CRUD operations from Django on the database.

Migrations were not present in Django versions before version 1.7. Let’s learn how developers dealt with the situation.

Problems Solved by Django Migrations

When migrations were introduced for the first time in Django, it became a huge hit. Django’s popularity grew exponentially. This proves migrations are a very important feature.

So, when there were no migrations in Django, developers had some difficulties.

They had to define the tables and database schemas according to models. Now, this may seem easy in listening but SQL is quite difficult. This became more complex when they had to create models with foreign key relations in them.

This also led to occasional errors and database inconsistency. The development was much slower than it is today. The developers would also have to keep track of all the changes they made themselves which was very difficult.

The process of reverting back was the same time-consuming experience.

These were some of the many problems faced by developers with no migrations.

Migrations resolved all of them and also some additional features were provided. Let’s learn in-depth about these features and solutions.

Django Migrations

In the previous section, we learned that SQL is used to create a Database Schema. That has to be the same as Models that we defined in our application. Writing SQL with the same constraints as our Models is difficult. It requires a decent knowledge of SQL.

Django solved this problem with the introduction of Django ORM or Object-relational Mapper. An ORM as its name will map the Model Fields to Database Table Fields/ Columns.

The model has 3 fields namely date, price, volume. These are defined in Python with model class attributes. Django ORM generates the table. It also defines corresponding fields and constraints.

The example table has fields mapped to the table. There is an extra field that works as a primary key. This is an awesome feature of Django ORM. It defines a primary key automatically if we haven’t defined our own. This has saved a lot of developers from database errors.

Django ORM does this via migrations. So, what are migrations?

Django migrations are nothing but Python Files. They contain Python code which is used by Django ORM to create and alter tables and fields in the database.

Django implements Models and changes to them on the database via migrations. Django generates migrations automatically. This is a key feature and you should also take care of these. Migrations are not meant to be corrected in normal conditions.

You can modify them but the file is accurate most of the time. We have used migrations every time we used a Model.

Key Benefits of Django Migrations

Migrations had a huge impact on the development process of web apps. They have greatly reduced the errors in one of the most error-prone areas by migrations.

This has made it easier for beginners to work with a database with no experience in SQL or databases. It has also increased the utility of Django and made the developer’s life easier. Some of these main benefits of migrations are as follows:

  • Creating tables without SQL

We have used MySQL and phpMyAdmin which is a graphical tool for databases. If we had used SQL instead, then making tables would become more complex.

For beginner developers, this could pose a real problem. Migrations had made Django more fun to use. It lets us bypass the whole SQL writing and makes tables automatically.

  • Models and Database Schema are always synchronized

This was the most error-prone zone for backend developers. Since they had to make the Database Schema same as the Models. It would take complex SQL and thus, resulting in errors. Sometimes, they can be real security threats but this is not the case with migrations.

Since migrations generate SQL automatically, it is always correct. That SQL contains the same constraints, we define in Models. We can see that SQL is generated by Django ORM in a later section.

  • Don’t Repeat Yourself (DRY) philosophy

DRY philosophy is one of the core principles in which Django is based on. Since Django is a framework for rapid development, this feature is an essential pillar.

Django migrations automatically generate SQL. This means developers won’t need to write the same code again. Not repeating the code makes the whole development fast.

  • Easier Version Control for Database Scheme

Migrations are Python files. Django generates a new migration file every time we modify a model. These files are not deleted and stored inside the app directory.

These files contain very simple codes. It is easy for a developer to understand what changes were made. This also makes it easy to revert back to previous Models.

Django migrations have several benefits, these were some important ones. We can now learn some very basic operations on migrations. We have used most of them in our previous Django tutorials.

Common Django Migration Operations

These are some of the common Django migration operations which you must know. These commands will give you an edge in handling Database Management System (DBMS). Migration is a trending topic for Django interviews. You will surely find this knowledge worthwhile.

Setting up Application

We will make a new application to implement these operations. In your Terminal/ PowerShell prompt, execute this command to make a new application.

python manage.py startapp Cars

Terminal Screen Display:

setting application - django migrations

You will have to install this app in settings.py. Add this app’s name in INSTALLED_APPS list.

installed app list

Now, in Cars/models.py file paste this code:

from django.db import models
#DataFlair #DataBaseMigrations

class Specs(models.Model):
    name = models.CharField(max_length = 20)
    price = models.DecimalField(max_digits=8, decimal_places=2)
    weight = models.PositiveIntegerField()

Code Display:

model for cars - Django Migrations

We created a very simple model for cars. There are 3 fields namely name, price, and weight. Now we will implement all the operations on this model one by one.

Note:

It is important that your Django project has a database configured and running. To configure database settings, please visit the Django Database tutorial. There you can easily set up the database and proceed with this tutorial.

1. Creating Migrations

We have done this process, every time we use a model. We will now create the migration files which will then be used by the ORM to create tables. Just run this command:

python manage.py makemigrations Cars

Terminal Screen Display:

Creating Migrations terminal screen

Thus, we have a Migration file ready. The name of this file is 0001_initial.py. Django automatically serializes the migration files with a numerical value. This file contains the migration class.

Although, at our level, we need not do any modification on it.

2. Applying Migrations

We created migrations in the previous file. Now, its time to apply them. We need to understand the difference. In the previous section, we just created some Python files.

These files will not affect the database until we apply them.

This command will apply migrations in the database.

python manage.py migrate Cars

Terminal Screen Display:

Applying Django migrations

This command can also take the name of a specific application. Django has made it so easy for us to apply migrations.

Migrate command can apply all the migrations at once if no parameter is given.

It is special because it will not apply unchanged migrations. So, if developers had changed only a particular model, they can simply write migrate.

It is important that you execute makemigrations before migrate command. Since a migration file is required before applying it to the database.

Now, we can check our database for this table.

phpmyadmin car_specs

There we have a table of cars_specs. If you check the structure, you will notice one extra field – Id field.

id field - Django Migrations

This is created automatically by Django ORM for the purpose of a primary key. We can define a primary key in Models for a particular field. It’s the default behaviour of Django ORM.

Every relational database requires a primary key in a table, otherwise, your table is not created. Django creators knew this very well and thus provided us with migrations functionality.

3. Editing Models

Website growth is not linear. As the website grows, there will be changes in Database Schema. So, when we change our models, we will again execute these commands:

makemigrations and migrate in their respective order.

It is necessary for the Model definition to match the Database Schema. Otherwise, we will get an OperationalError.

Now, we will change the specs model. As weight is not always an integer, we change it to decimal.

Paste this Code in cars/models.py:

from django.db import models
#DataFlair #DataBaseMigrations
# Create your models here.

class Specs(models.Model):
    name = models.CharField(max_length = 20)
    price = models.DecimalField(max_digits=8, decimal_places=2)
    weight = models.DecimalField(max_digits=7, decimal_places=3)

Code Display:

editing models - Django Migrations

To apply this change, we will again run the commands.

database table Django migrations

This should be the output. Let’s see the database table.

database table

As we can clearly see, if we had to change the same manually, it would have been a time-consuming affair.

4. Reverting Migrations

We all love the undo option. It reverts back the previous state and saves our time. Django also gives us an option to revert back. Suppose you are working on production site and made some change to the Database Schema.

Change may not yield positive results as you were expecting. Then, it will be time-consuming to change the Models again as the previous schema. Also, this can be more prone to errors, if you do it manually. Django allows you to revert back to any previous migration. The feature is important as it will change the Database Schema. That means you don’t need to rewrite anything.

This can be easily achieved with migrate command. This time you don’t need to perform makemigrations command since the file already exists.

Now, use this command:

python manage.py migrate Cars 0001

Terminal Screen Display:

reverting migrations

When we ran this command, we passed some parameters. The first parameter is the application name. In this case, it was Cars. The second parameter is the migration file name.

We passed a number 0001, then the name of the file. This is one of the cool features of Django. It will match the file with the matching string 0001. Since it is a primary key for Django, therefore there are no conflicts. You can also use a complete name, the result will be the same.

Let’s take a look at the database structure now.

The weight field has again reverted back to an integer.

Note:

Although I compared undo operation with reverting migrations; it is not the same. When we apply another migration, it will revert back the Database Schema. That is the only part which is undone.

Suppose, you have added some fields which were not there in previous Models. Now, when you revert back the schema changed, the data which was saved in the new fields were deleted. This is the difference. Therefore, always be vary when modifying databases.

When we reverted from this migration, the file was not deleted. The migration files are never deleted by Django. So, you have to be careful when applying migrations again. migrate command will automatically apply last migrations of the app.

You can either delete that file or apply the migration to apps specific.

5. Indexing all Migrations

When applications became too big, this operation comes in handy. It will list out all the migrations, existing in the Django project. We can be specific to applications like in migrate command.

The command:

python manage.py showmigrations Cars

Terminal Screen Display:

indexing migrations - Django Migrations

This command will show all the migrations if no parameter is given. The Cars application has 2 migration files. Their names are listed out.

You can notice that [X] in front of the 0001_initial file. The X represents that migrations have been applied. We just reverted the migration to 0001. The second migration is unapplied, therefore [ ].

It helps to determine which migrations are applied in the current database.

This command is mostly used when you are working in teams. Since many people are applying migrations, a developer can easily check which migrations exists and work accordingly.

6. Naming Migrations

Migrations can be a user named too. This operation will let you provide meaningful names to migration. It is used at the time of creating migrations.

The command:

python manage.py makemigrations Cars -- the name change

Since we are not changing anything in our models, we will delete the migration file.

  1. Go to the project directory.
  2. Inside it, open Cars/migrations directory.
  3. Now, delete the 0002 files.

You have to do this manually.

Now execute the above command.

naming migartions

Now, you can check that we have a new migration named 0002_Change. It is not applied yet.

Django keeps track of all the changes in Models. So, if models are not changing, it will not make a new migration file with the same code. This decreases code redundancy and improves performance.

Django keeps this information in a table in the database. The table is named as django_migrations.

This table is key to Django’s perfect migration management. You can see the last applied migration in the table.

show migrations - Django Migrations

This table gets updated as to which migrations are applied. The showmigrations command checks from this table for applied migrations.

These are all the basics that you will need to work on migrations.

Summary

Now, we can conclude that migrations are a very important feature of Django. They made rapid development possible with ease. Django provides us with many commands and operations to easily manage and apply migrations.

We also learned that un-applying migrations are not the same as undo as the data can be lost in the process.

Hope you enjoyed this tutorial. Comment for any queries. Keep Learning😀

You give me 15 seconds I promise you best tutorials
Please share your happy experience on Google

follow dataflair on YouTube

2 Responses

  1. kathan patel says:

    nice document

    how can i connect crate.io database plz help me

  2. Karan Mittal says:

    Hey Kathan,
    Currently, Crate.io is not supported officially by Django. So you won’t get the built-in support. If you want you can use crate.io with django. but you will lose most of django orm’s functionality with that.
    Crate.io do have a python binary. That can make your work easy. if a binary exists, you can easily connect to the database. Although, if storing complex data you will have to learn the SQL for queries and Database operations.
    Thanks for linking the content

Leave a Reply

Your email address will not be published. Required fields are marked *