Configuing Automatic PostgreSQL Database Backup Using Cron

First, use crontab,

$ crontab -e

add the following lines at the end of the file

 ####backup toppers sb everyday##########  
   
 # m h dom mon dow             command   
   
   0 0  *    *   *   cd /home/user/db_backup && ./backup > /dev/null 2>&1  

where,

 # m h dom mon dow  command   

is the format.

m - Minute (0-59)
h - Hour (0-23)
dom - Day of Month (1-31)
mon - Month (1-12)
dow - Day of week (0-6, starting with Sunday)

and the command is the shell command that you want the cron to execute at the specified time.

Here in the above example I've asked cron to execute the command

 cd /home/user/db_backup && ./backup > /dev/null 2>&1  

at every dom, mon and dow. That is what for the '*' is used. So, as everyday starts, cron will run the command I've specified.

In the above command in cron, I've asked cron to execute a shell script named backup. I've done this in order to declare some variables or settings needed for backing up the database without complicating the crontab. It will also be easy to maintain different shell scripts for backing up different databases with respective configurations.

In backup file,

$ nano /home/user/db_backup/backup

 #! /bin/bash  
 FILE="/home/user/db_backup/backup_$(date +%Y%m%d_%H%M%S).pgsql"  
 pg_dump --no-password database_name > $FILE  

I've created a file name and stored it into FILE variable, then dumped the database into a new file with name FILE. If you are using openscg or something, and if that specific environment is not added in the current environment, then add the following line next to shebang,

 source /opt/postgres/9.1/pg91-openscg.env # for openscg  

But, the above setup will end with an error message,

pg_dump: [archiver (db)] connection to database 'database_name' failed: fe_sendauth: no password supplied

as there should be a password supplied in order to authenticate a user to back up the database. As we are automating the backup we can't (I'm not sure here) enter the password interactively every time. So, we've to supply the password in some implicit way. Postgresql gives us an option in the form of a .pgpass file.

.pgpass file should be created in home directory and it should contain the following details,

$ nano ~/.pgpass

 hostname:port:database:username:password  

now restart the postgresql and check if the backup works without a password by executing the backup file.

$ chmod +x /home/user/db_backup/backup  # making 'backup' executable
$ /home/user/db_backup/backup

This will backup a database in the db_backup directory or any other directory you've specified in backup script file with current date and time.

Creating symbolic links without "too many levels of symbolic links" error

Lets create a shell script named 'what' with a single line, no! two lines with shebang included.

1:  #!/bin/bash   
2:  echo "nothing!"  

and save the file.

Now, make it executable.

$ chmod +x what

then create a symbolic link for it in bin or any other directory that is in your PATH variable.

[
  to check the path just echo it,
  $ echo $PATH
]

$ ln -s /home/user/what /bin/what 

here, you could notice that the absolute path of the file to be symlinked is used with 'ln'. It would be sometimes tempting to use relative path or you might feel lazy to type in full path. Nevertheless, always use absolute path while symlinking. If done with relative path, here for example. While doing,

$ ln -s what /bin/what

then, 'what' would be symlinked to /bin/what. i,e I can now run 'what' script from anywhere in my system just like we use 'cd', 'ls' and so. Now when I try to execute 'what', the symlink (/bin/what ) will again point to 'what' (the command) and again it will go run the symlink /bin/what and again from symlink it will point to command 'what' and the cycle continues. This will be indicated as error message in shell as,

zsh: too many levels of symbolic links: what

With Forth - 1

I've started learning the Forth programming language. If you are about to learn Forth, then pay attention when they say "It's a stack based programming language". At first, I thought this as some sort of technical shit boasting out the implementation of Forth or something as I'm a low life programmer. But after fiddling with it for a while (thanks to the "Implicit Data Passing" section of Thinking Forth by Leo Brodie) I finally understood what stack based programming is and now I've written my first working program in Forth which I was trying for the past few hours: a function to sum two numbers.

: ADD + ;
5 10 ADD CR .

Now, this stack based programming is exciting (I'm yet to know thier pros and cons). Here, in the above two lines of code, I'm assigning the '+' operator to the word 'ADD'. Then in the next line I'm adding the numbers 5 and 10 then printing them on the screen as the word 'ADD' means '+'. The second line of code is  equivalent to

5 10 + CR .

where 'CR' is carriage return and '.' is used to print the output value to screen. So, don't worry about 'CR' and '.'.

What I understood from the above: 5 is first stored (pushed) in to the stack and  then 10 is stored (pushed) above 5. With Last In First Out LIFO 'ADD' which  translates to '+' adds the last two values (by popping them out) of the stack and  puts (pushes) the output on the top of the stack. So, here we would be having 15  at the top of the stack which is printed out to us by '.' operator.

Now, I wanted to show how this program works to my friends by summing a pair of integers. But, it was painfull (& not so cool) to type 'CR .' at the  end of each function call to print the output of the function 'ADD' each time. Then  the below lines from Thinking Forth, by Leo Brodie, flashed into my brain (which won't happen very often),

The smallest atom of a Forth program is not a module or a subroutine or a  procedure, but a “word.”

and

Everything in Forth is a word.

So, here '+' should be a word, 'CR' should be a word and '.' should be a word too. And in Forth we can group together any number of words and assign it to a word.

: ADD + CR . ;
5 10 ADD

This is not only cool, but seems powerful to me. Now, I'm getting excited to  further dive into Forth.

Also, some of my friends argued that the below piece of code won't qualify for a function to which I wrote the below code, a function to sum four numbers.

: 4add + + +  cr . ;
9 8 1 2 4add

Which outputs 10

Installing and using a different Python distribution of our choice in shared hosting - server

Every step below need to be done on server through ssh. So, if you don't have an ssh access to your server then contact your service provider to activate it.

I hope you selected a Python version to install already. If no then do select one based on your needs and download that particular Python distribution source code from Python official site using wget or curl directly into your server. If you don't have the above tools to download then download the source in your local system and use SFTP or SCP or by any other means and put them in your server.

Here I'm downloading the Python 2.7.8

$ wget https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz

Then unzip the file

$ tar -zxvf Python-2.7.8.tgz 

The above command will unzip the source code into a directory named Python-2.7.8. Now going ahead into the source code directory,

$ cd Python-2.7.1
~/Python-2.7.8$

create a directory where you want your Python to be installed.

~/Python-2.7.8$ mkdir ~/.localpython

Now we are ready to install our new Python distribution.

Installing:
Lets configure the Python to install in our custom path by passing our custom installation path through command line argument 'prefix'.

~/Python-2.7.8$ ./configure --prefix=/home/user/.localpython

In the above command replace the user with your user name or replace the whole prefix path with the installation path of your choice.

Then run make,

~/Python-2.7.8$ make

Skip this section if you already have 'make' installed in your server [most of the case]:

If you don't have 'make' already installed(only for deb based systems), then you could install a local version of it. 

Download it from here.

Then  create a directory to install make in local environment

$ mkdir ~/.localmake

Then install the make from downloaded deb file as

$ dpkg-deb -x make.deb /home/user/.localmake

It will install the make application inside the ~/.localmake and now you could use this make for our Python build by calling the make as,

$ /home/user/.localmake/usr/bin/make

Then finally, installation:

~/Python-2.7.8$ make install

[ ~/Python-2.7.8$ /home/user/.localmake/usr/bin/make install ]  #for users with local make

Now, we can check our new Python distribution by running

~/Python-2.7.8$ /home/user/.localpython/bin/python
Python 2.7.8 (default, Apr  3 2015, 15:18:55)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 

The above command will open up our installed Python distribution which it will print in the first line [version].

We have successfully installed our required Python version/distribution. Yet, we have to solve another issue.

Every time we wanted to use our Python version, we need to call it with the full path of the binary, as the default 'python' command would open the default Python version that was installed system wide. In most cases, we are about to run Python applications by just calling the command 'python' and it's the default method for most programs too. So, lets solve it by creating a symlink for python for our installed version.

create a local bin directory,

$ mkdir ~/bin

Then create symlink ,

$ ln -s ~/.localpython/bin/python ~/bin/python
$ ln -s ~/.localpython/bin/python-config ~/bin/python-config

Also, some of the python programs call python with 'python2.7' command, so lets create a symlink for it too.

$ ln -s ~/.localpython/bin/python2.7 ~/bin/python2.7
$ ln -s ~/.localpython/bin/python2.7-config ~/bin/python2.7-config

Now, add ~/bin to our PATH,

$ export PATH=~/bin:$PATH

Then test if everything works fine by opening Python interpreter.

$ python
Python 2.7.8 (default, Apr  3 2015, 15:18:55)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

and

$ python2.7
Python 2.7.8 (default, Apr  3 2015, 15:18:55)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

or by issuing the command,

$ which python
 /home/user/bin/python

Add this path permanently by putting it into ~/.bashrc file so that it will work on every further sessions.

$ nano .bashrc

and add the below line and save it with CTRL+x.

$ export PATH=~/bin:$PATH

Finally activate the modified .bashrc file

$ source ~/.bashrc

Installing pip:
Download the Python script get-pip.py and run it.

$ wget https://bootstrap.pypa.io/get-pip.py
$ python get-pip.py

The above script will automatically download and install pip and setup tools for us. Once it has been installed you could install whatever the packages you need using pip. Before that let's check,

$ pip freeze

Django: using a Model from an app as a Foriegn Key to another app's model

Rich is a model in an app named strain. I have another app in the same Django project called grain which has a model named Poor. I now want to use the model Rich as a foreign key to the Poor model.


We shouldn't import the model directly as it will lead to circular import dependency.

A short note on circular import dependency:
Lets assume that we have two modules in a same directory each importing each other.

module 1 -> a.py

from b import dexter
# rest of the file

module 2 -> b.py

import a

def dexter():
    #dexter function

# rest of the file

If we try to run the b.py module then it will try to import the module a. During the process of import, the a.py will try to import dexter from b.py which in this situation wasn't defined yet as we are still executing the first line of b.py. So no dexter function will be found to import leading to an error.

Lazy Relationship is what we wanted!
We shouldn't explicitly import the model to avoid circular import problems.

grain>models.py file

class Poor(models.model):
    rich = models.ForeignKey('grain.Rich')
    # remaining of the class

This simply solves our problem.

If we already have a Poor and Rich models with data in them we should take care of them first before migrating our database to new changes.

Fork and sync a project from Github

Forking a Github project is very easy as it is 1-click work, if that project is in public domain. Then cloning the forked project again is very easy with the git clone [$ git clone  https://github.com/user/project.git] command. Even, the non git users of a large volume might have used the above command for various reasons.

After a [few] commit[s], if we think that the cloned/forked copy of the main project is solving any issues or adding any new compelling feature [after running all the necessary tests] then making a pull request is also 1-click [followed by a few clicks] work.

 In pull request, our modifications or code additions will be reviewed [best place to learn things] and any more modifications necessary will be pointed out to us. The chances are there even to get our pull request closed as someone else might have made a pull request that solved the issue in a better way than yours or due to million other reasons.

No matter what, I wanted to have my forked project in sync with the original [upstream] project and so does many developers. Here came the need for me to get to know about 'remote' [1]. So I did.

First checking remotes if any exists with the current repo.

$ git remote -v

If the upstream or the original repo from which the project have been clone wasn't present then add it [2],
  
$ git remote add upstream https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git

Method: 1 

Then do fetch the latest changes from the upstream [3],

$ git fetch upstream

merge them with the local repo [it is advised to push/pull the local repo with your own remote repo and have them in sync before syncing with upstream]  and resolve any conflicts if exists.

$ git checkout master

$ git merge upstream/master

Method: 2 [4]

$ git pull upstream master




pull does both the fetch and merge at a single step. Then do,

$ git push origin 

to sync with your very own remote branch.

References:
[1] https://help.github.com/categories/managing-remotes/
[2] https://help.github.com/articles/configuring-a-remote-for-a-fork/
[3] https://help.github.com/articles/syncing-a-fork/
[4] http://stackoverflow.com/a/4936482

Django (Python): RuntimeWarning: SQLite received a naive datetime while time zone support is active

Problem: when running the command,

C:\Users\What\Dropbox\Django\mysite>python manage.py shell

encountered a warning followed by a lengthy traceback from Python interpreter:

C:\python27\lib\site-packages\django\db\backends\sqlite3\base.py:58: RuntimeWarning: SQLite received a naive datetime (2014-05-27 02:45:18.709000) while time zone support is active. 

....

.....

....

Solution that worked for me: Opened the settings.py file from the current project and replaced the TIME_ZONE variable,

TIME_ZONE = 'UTC'

to

TIME_ZONE = timezone.now()

and added the line

from django.utils import timezone 

at the top of the file to import the timezone.