Python

By and large, python is the default tool for data science. Every data scientist I know uses python. However, this is a bit of a misnomer. Yes, they all use python, but this is more in the sense that every writer uses paper to write on.

Python itself, the base it provides, is rarely directly touched by the data scientist. Instead, they use libraries to handle with prewritten code to abstract away coding details. To not go to far into details right not let’s start with the basics.

Where can you get Python?

Python is a programming language. Historically, this would mean that Python defines a programming syntax but does not provide a compiler which can translate the textual code into machine code (zeroes and ones) for the machine to execute. However, these days it has become common for many programming languages to provide compilers. Python is such a great modern programming language. You can find it on python.org.

How do I use Python?

Python is a programming language. You can write code in text and your python compiler can translate that code into executable code and run it on your machine. There are a host of tools which can help you along with programming python, but I think it’s good to know how to use python out-of-the-box.

Odds are that if you installed python on your machine that it is directly available through your command line. That’s your cmd.exe on windows and ctrl+alt+t on ubuntu. I’m not sure about Mac. On any other OS my guess is that you already know the command line.

In windows you can enter the following command to enter the python environment: py

In other OS’s this is likely: python

Given that windows is the exception I will use python as the default. Just remember that as a windows user you will need to py instead of python in any of the command line examples.

If you have written python code in a text file you can execute that code with: python <file>, where you replace <file> with the name of your file. Remember to include the extension of the filename as some OS’ hide these.

There is some difference in behavior between running python without arguments and python <file> with a file as argument. The first command leads you to an interpreter where you can write and compile code live, whereas the second executes the code in your file and then returns to the terminal. If you have written some python code and want to execute it leading into the interpreter afterward you can run: python -i <file>.py. This is great for debugging.

Minimal Working Example

If you’ve installed python this would be a good point to get our fingers dirty and write some base code.

Open a terminal and check if you can use the python or py command. If not there might be something wrong with your installation.

You’ll recognize when you are in the interpreter if you see the terminal screen change to something like this:

Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Let’s go over the lines you see here quickly. The first tells you which python version you are running. In my case, python 3.8.10. It also shows the build time, which was back in november.

The second line shows the compiler that python builds to. To be very brief about this. Python doesn’t directly compile to machine code, instead it translates your python code to C code which is then compiled to machine code that the machine can read1.

The third line gives you some helpful information, such as the help command. This gives you access to the documentation. You can run it as help() to enter the documentation. Type quit to exit it again. You can also access specific parts of the documentation immediately, for example running help(str) gives you the documentation of how to manipulate strings. This can be a very powerful tool during code development, though often the documentation is more readily accessible via the internet.

The fourth and final line is the current interpreter line. You can type here as you would in a texteditor. When you have completed code you wish to execute. You press enter and it will run that code for you. A small note though before we continue. The arrows >>> mark the beginning of a code block instead of a code line. You can also get dot marks ... which means that your current code is still part of the code block and not yet executed. Inevitably there will be a moment where you are perplexed that your code isn’t running usually because you are still in a code block and haven’t properly exited it yet.

Now we have that out of the way, let’s run our first code! Copy the following line in your interpreter and press Enter:

print('Hello World!')

Note the output. It may not seem like much, but you have just printed the text Hello World! in you interpreter. You can run help(print) to find the technical documentation for writing text to your screen.

It may not seem like much, but this function is your first tool to communicating with your program and especially inspecting what your program is doing.

Now we have a minimal understanding of the interpreter let’s investigate how we can run code written in text files. This is the main way people interact with code so it is essential to know how to do it.

First, we exit our interpreter. We can do this by copying running the following command:

exit()

Now navigate to a directory which you want to use for testing purposes. For example, you can create a new directory with mkdir TestDir and then enter this directory with cd TestDir.

Let’s create plain text file in TestDir. For Windows this can be done with Notepad. In Ubuntu this can be done with Gedit. Write the following text in your text editor:

print('Hello World!')

Now save the file as hello_world.py2. Make sure it is located in the TestDir directory we created earlier. Check again that your terminal is in the same directory as where we created the file. We can do this by running dir in Windows or ls in Linux. This command prints the contents of the current directory. Our hello_world.py should be listed here. Windows users may find that hello_world.py.txt is listed here instead. Don’w worry, just use this name instead of hello_world.py when we run our scripts.

And then, finally, the moment has come. We run our script by entering in your terminal the command:

python hello_world.py

Did it work? You should get the output Hello World! in your terminal. I’m going to assume it did work. Congratulations! You’ve written your first python script. If on the other hand it didn’t work, check you error messages and try to find where it went wrong.

A very useful last thing to check out is to run the script again but this time with the additional option to enter the interpreter after completion. To do this run the command:

python -i hello_world.py

Notice that now Hello World! was printed followed by a new line which we are free to execute code in. This enables us to write some custom code to continue after running the script. I find it useful to inspect the state of the code after running, but I’ve also used to run some live where I just need to try a bunch of things and don’t want the hassle of saving it continuously.

What can I do with Python?

We have created a very simple print example. As of now the sky is the limit. With access to the python interpreter you can do almost anything on a machine. We can write some code to calculate Fibonacci numbers:

def fibonacci(n):
    if n < 0:
        return(0)
    elif 0<=n<2:
        return(1)
    else:
        return(fibonacci(n-1)+fibonacci(n-2))

fib10 = fibonacci(10)
print(fib10)

We can write create loops for some mental support:

import time

quotes = ['You are doing great!','Keep it up!','You deserve a break!','Enjoy :)']
count = 0
N = len(quotes)
ten_seconds = 10 #seconds
while True:
    quote = quotes[count]
    print(quote)
    count += 1
    if count==N:
        count=0
    time.sleep(ten_seconds)

If you get stuck in the interpreter on running this code indefinitely use Ctrl+c to interrupt the code. If you get stuck on running this in the terminal, just close out the terminal. This should kill the python program as well. Luckily it is not very resource intensive. Even if it were to run forever it would not bother you except with perpetual messages.

Or maybe you want to do something more beastly like training neural networks. I’m actually not going to give you the code for this, mainly because running it wouldn’t make your code work as you are probably missing essential libraries.

The Main Benefit

So, we have seen a bit of python, but why Python over other languages? Also, why is Python so much more popular than other programming languages? Python is an excellent programming languages for a couple of reasons:

  • It’s syntax is relatively easy to learn and easy to get started
  • It has a vast and easy to use package manager in pip which allows you to install other libraries.
  • It is free and open source.
  • It has a huge and I mean HUGE community with tons of forums, code examples, projects and anything you’d like from a community.
  • It has great IDE’s (editors) which can help you writing your code.

Why not Python?

  • Not very suitable for GUI design. Improvements are being made but languages like Java are much more suitable. Also, you may be able to run your python script from within a program or language that helps does work well with creating GUI’s.
  • Not very fast. You can create basic games for python, but high-end games usually run on C# and the like. Performance difference between Python and C is roughly a thousandfold. For most use cases this performance difference does not matter, but sometimes you need that speed and it can be difficult to get that from Python.
  • Security motivations. Python is not unsecure, but almost everyone runs other libraries using python. These other libraries is where security issues can become a concern. Bigger libraries like NumPy and PyTorch have their own means of validating security, but smaller packages especially those written by just one or two individuals may not be the safest3. There are however not many languages out there which offer better control of their packages4.
  • Specialization. Python is a general language that relies on libraries and packages to offer the specialization you need. This means that you are commonly looking for a package which can solve a problem for you (hence this blog series). On the other hand, there are many languages which specialize on a specific field. R is a good example of this. R is capable of many things which Python is also capable of, but R is specifically developed for statistical analysis. This is where R can and often has outpaced Python’s libraries.

Conclusion

  1. Remember how computers are essentially thinking in ones and zeros? Well, I don’t speak binary and most programmers only speak rudimentary binary. So we rely on translators (a.k.a. compilers) to translate our code to ones and zeros. As far as I am aware the succession for my machine is Python > C > Assembly > Machine executable code. C and Assembly are programming languages of their own, but each level abstracts away some patterns to make it easier for the user to use. This comes with the cost of some inefficiency. As a common rule of thumb, the closeness of your programming language to machine code determines the execution speed of your code. The trade-off is commonly developer time versus running time. ↩

  2. Normally save python scripts with the extension .py, but it isn’t necessary to do this. We can also save it as .txt or any other extension. As long as it is a plain text file our python compiler can attempt to compile the code. And yes, it can also try to compile your local copy of plain text Shakespear. It will mostly likely run into an error on the first line, but nothing is keeping you or your compiler from giving it a go. ↩

  3. A library for Java called log4j caused a huge raucous when a security vulnerability was found. The library itself was maintained by just one or two individuals, despite the package being used in almost every Java application either by directly in the application or other libraries building on top of log4j. It was a serious headline in the security community. ↩

  4. Payed for languages may claim to have better security but consider that they have to have had to develop their code by their employees. This severely cuts the amount of development time their code gets. Contrarily, the larger packages in open source communities are created by the same experienced coders which work for closed source companies and now there are simply much more of those coders having a look at the code. The openness of these communities also means that you get to hear whenever a vulnerability is found, whereas with closed source code you have no idea how many issues there are and how diligent the company is to solving them. You are completely reliant on their word instead of having the ability to verify the security yourself. ↩