Beginner

The easiest and simplest mechanism to store data from python is the humble file storage which is often, but does not have to be, text based.  There are no libraries that you require, and you can use native python functions to open and write to the file very easily.

There are many use cases for file storage and is usually the “go to” method when hacking a quick solution or prototype together.  These are also arguably good solutions for production use cases.  

Overview of using storing data to files in Python

The typical use cases has the following commonalities:

• Setup: There’s no setup that is required for files.  You can create the file even from python
• Volume: Size Small-ish file size (< 5-10mb).  You can go larger of course if your application is not doing heavy reads or writes nor if it doesn’t require fast response (e.g. batch processing)
• Record access: Does not require to search data within the file to extract just portion of the records.  You would load or save all the data in the file in one go
• Data Writes: You can either append to the file or you can upload and download all data in the file.  
• Write reliability: You do not need to have multiple writes at the same time – there is only possibility (or likelihood) of one person writing at one time, and if there was a case of multiple people writing at once, the consequence are not serious for your application.  There are ways to put a lock on a file to prevent conflicts, but you should double check if a file is the write option for you
• Data formats: You may have structured record based  (such as comma separated value – CSV or tab delimited) or unstructured (eg document of text or JSON format).  You can also store binary data in a file as well – e.g. for images
• Editability: You may want or allow direct editing of the file by other applications or direct editing by people 
• Redundancy: There’s no inbuilt redundancy.  If there is any failure (data corrupt, the server with the file fails), then you’re out of luck.  You need to setup your own mechanisms (e.g. replicate file to another server automatically)

Code examples to read and write to a file 

Here are two sets of example code for writing and reading from a file.  It is very easy and does not require any libraries.  The one thing to be mindful of is what mode you want the file to be opened- read, write, read and write.

Open a text file for (over)writing:

To write to a file, it’s very easy to do so which is to use the ‘w’ switch on the open() function.  There are other options as well:

• ‘r’ – Reading
• ‘w’ – Writing to a file
• ‘a’ – Append to end of file
• ‘r+’ – Read and write to the same file
• ‘x’ – Used to create and write to a new file

file = open( ‘population.txt’, ‘w’)
file.write(‘Japan’)
file.write(‘United States’)
file.write(‘Australia’)
file.write(‘China’)
file.close() #file is released and closed

You will then have the following output file of population.txt:

Japan
United States
Australia
China

Open a text file fully for reading:

Using the same population.txt file created above –

file = open( ‘population.txt’, ‘r’)
data = file.read() #read full contents of file into a single string
file.close() #file is released and closed
print(“*** file start ***”)
print( data )
print(“*** end file ***”)

The output would be:

*** file start ***
Japan
United States
Australia
China
*** end file ***

Now to explain this a bit further, the open() command helps to open a file where you need to specify how the file is to be opened – in this case with ‘r’ to indicate it is for reading.  There are other options as well:

• ‘r’ – Reading
• ‘w’ – Writing to a file
• ‘a’ – Append to end of file
• ‘r+’ – Read and write to the same file
• ‘x’ – Used to create and write to a new file

Read a text file line by line:

file = open( ‘population.txt’, ‘r’)
data_list = file.readlines() #read full contents of file into a list of rows
file.close() #file is released and closed
print(“*** file start ***”)
counter = 0
for row in data_list:
  counter = counter + 1
  print( f”{counter}:  {data_list}” )
print(“*** end file ***”)

The output would be:

*** file start ***
1: Japan
2: United States
3: Australia
4: China
*** end file ***

The difference in above to the first example is that the data comes out in a list separated by a newline so that you can process each row.  Please note, you can simplify the above using the enumerate to avoid having the separate counter variable setup.  E.g.

print(“*** file start ***”)
for index, row in enumerate(data_list):
  print( f”{index+1}:  {data_list}” ) #Note that when using enumerate, first index is 0
print(“*** end file ***”)

Summary of writing and reading to a file

Reading and writing to a file is a very straightforward native operation in Python. There are many other related operations that you can do ranging from putting a lock on a file to prevent two processes writing to the same file, checking file attributes such as access and size, and many other operations.  At the most basic though, you can simply use the “open” statement to do the read/write to satisfy most of your needs.