Bug Repellent

If you work as a software engineer in a team, you’re probably familiar with a piece of software called Git. It’s probably a part of your development workflow. You type a few magical incantations and somehow your code is versioned and distributed to your colleagues. Occasionally, something goes wrong and you type a few more commands and if you’re lucky, things sort themselves out.

My goal in this blog post is to help you get Git (try saying that five times fast), to understand what is going on under the hood as you execute a variety of common commands in your Git workflow. But before we go under the hood, we have to travel to the past and learn a little bit about the history of version control systems. This history isn't going to cover every tool, just the ones that I think are indicative of the technological changes in version control systems.

We’ll start by traveling all the way to Bell Labs in 1972 and the dawn of one of the first version control systems, Source Code Control System. The architecture of the tool consisted of three different parts: a delta table, control and tracking flags, and a set of control records. The delta table, as you might expect, is a table that stores each of the changes made to a file. Control and tracking flags were used to set permissions and control releases. And control records were used to keep track of when lines of code were deleted or inserted into a file by storing those insertions and deletions into special records. In this way, this version control system pioneered some of the early principles that we'll come to see in later version control systems.

SCCS, as it was known, was popular until the year 1982 when its successor, Revision Control System came to prominence. The RCS system was not distributed at all, so it wasn't possible to store copies of the code that you were versioning on a central server or another machine. Multiple people couldn't edit the same file at the same time, so merge conflicts weren't really a thing that happened. It had one simple job: store different versions of code.

Next, we'll travel to 1990, and the release of the Concurrent Versions System, commonly known as CVS. Unlike RCS, CVS employed a client-server model. A copy of the repository was stored on a central server and several clients could make copies of it. At this point in time, it was possible for multiple people to be editing the same file, so it was possible for two individuals to make conflicting changes. To work around this issue, CVS required that you fetch and merge the latest changes from the server into your code before making any commits.

Finally, we’ll travel to a little over a decade ago to the year 2005, where a tool that we’re all familiar with came to fruition: Git. Git was different from its predecessors in a lot of ways. For one, it didn't require you to be on the latest version of a file before making changes. You could make a change to a file and then pull in any updates that happened after you made the change. Furthermore, Git was distributed in a decentralized nature, Git repositories could exist in a first-class nature on developer's machines, GitHub's servers, your company's CI build, and so on.

Why did git get so popular?

It was for a couple of reasons, and everyone has a perspective on what those reasons were. Tools like GitHub certainly made Git a little more popular by providing a centralized space for developers to discover and share code. Git also had a merge strategy that was a lot easier to navigate than its predecessors. And finally, after development, Git was used as the version control system for the Linux kernel codebase, giving it an immediate large adopter.

People's opinions on the reasons for Git's rapid adoption differ but in any case, here we are. Most software teams use Git to version and collaborate on their codebase.

So how do most people use Git? Well, you've probably run a command like this to get the latest copy of the codebase from a remote server onto your machine. What did this command just do?

$ git clone https://github.com/nteract/nteract.git
Cloning into 'nteract'...
remote: Enumerating objects: 310, done.
remote: Counting objects: 100% (310/310), done.
remote: Compressing objects: 100% (98/98), done.
remote: Total 49241 (delta 218), reused 255 (delta 208), pack-reused 48931
Receiving objects: 100% (49241/49241), 16.00 MiB | 3.21 MiB/s, done.
Resolving deltas: 100% (34612/34612), done.

There's all this business about objects and deltas and compressing and enumerating and resolving and oh my goodness! There's quite a lot going on in such a few lines of standard output.

To dive a little bit more into this, we're going to need to poke into a directory that exists on every Git-versioned project: the .git directory. Here's what its contents look like in our newly cloned directory.

$ ls .git
HEAD            branches        config          description     hooks           index           info            logs            objects         packed-refs     refs

There's an objects directory in there. Let's poke around it and see if we can get a sense of what Git might've been enumerating and counting and compressing when we cloned our directory.

$ ls .git/objects/
info    pack

Hm. There are only two directories in there: info and pack. Let's dive into them and see what we can find out!

$ ls .git/objects/info
$ ls .git/objects/pack
pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.idx
pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack

OK! Now we're getting somewhere a little bit more interesting. The info directory is empty but the pack directory contains two files. One with a .idx extension and another with a .pack extension. Now, we could try to cat these files to look into their contents, but they're binary files so looking at that output won't be much help.

$ file .git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.idx
.git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.idx: Git pack index, version 2
$ file .git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack
.git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack: Git pack, version 2, 49241 objects

You can see from the output above that the .pack file contains about 50,000 “objects.” Thankfully, I spent some time looking into this and will tell you right now what these objects are. Hurrah for sharing!

Objects, in the Git context, consist of a type, a size, and some contents. There are four types of objects.

1. Blobs: An object that is used to store file data.
2. Trees: An that object that is used to reference multiple blobs or other tree objects.
3. Commits: An object that contains a reference to a particular tree, the timestamp on which a commit was made, the creator of the commit, and other metadata.
4. Tags: Annotated tags are stored as objects in the git. Similar to commits, they contain a timestamp, an author, and an associated message.

So what just happened when we cloned? Well, Git pulled all of the objects associated with our project: including commits, files diffs, and tags. That's what those 49,241objects that were pulled in from the server on GitHub were. Once they were pulled in, Git compresses them into a single packfile. How big is the compressed file?

$ du -sh .git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack
 17M    .git/objects/pack/pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack

It's about 17 megabytes. Those 17 megabytes contain every commit, the files associated with that commit, and every tag on the project. To get a sense of all of the objects that have been compressed into a Pack file we can run git unpack-objects -n.

Now that you've got a copy of the code base on your local machine, you'll likely make a new branch on which you'll start to make changes.

$ git checkout -b safia/my-new-branch
Switched to a new branch 'safia/my-new-branch'

Standard output says that we switch to a new branch, but what actually happened under the hood? To answer this, we will need to look inside the .git directory located inside every git-versioned repository.

$ ls .git
HEAD            branches        config          description     hooks           index           info            logs            objects         packed-refs     refs

Let's take a look at the contents of that HEAD file. If you're familiar with Git, you've probably executed a command like: git push origin HEAD to push your updates to a centralized server. What are we actually referencing there?

$ cat ./git/HEAD
ref: refs/heads/safia/my-new-branch

Let's see what's inside the file that the ref is pointing to here.

$ cat .git/refs/heads/safia/my-new-branch
e35102c15bd63698b6dcb721e161c4d630e2d6cc

Oh! We've got a hash in here. What is this hash referencing? There's a useful command: git cat-file that allows us to print out details about the object that is referenced by a SHA-1 hash.

$ git cat-file -p e35102c15bd63698b6dcb721e161c4d630e2d6cc
tree 6cf0113d017fc604de9481758fc6a578a4067dd2
parent aad3eac9629ee28c4d6030e1091e8089dee66cd9
parent b7d058a50b14ea9b45c128da2af2e260b0ef1ea1
author Kyle Kelley <rgbkrk@gmail.com> 1537647280 -0400
committer GitHub <noreply@github.com> 1537647280 -0400

Merge pull request #3341 from nteract/renovate/next-7.x

Update dependency next to v7.0.0

Cool! So it turns out that that hash is a reference to a tree object. In this case, the tree object is a reference to a set of changes under a commit.

So, we're going to make a change, stage it, and commit it. You might have heard those words used in the context of Git before. What do they mean?

$ tree .git
.git
├── HEAD
├── branches
├── config
├── description
├── hooks
│   ├── applypatch-msg.sample
│   ├── commit-msg.sample
│   ├── post-update.sample
│   ├── pre-applypatch.sample
│   ├── pre-commit.sample
│   ├── pre-push.sample
│   ├── pre-rebase.sample
│   ├── pre-receive.sample
│   ├── prepare-commit-msg.sample
│   └── update.sample
├── index
├── info
│   └── exclude
├── logs
│   ├── HEAD
│   └── refs
│       ├── heads
│       │   ├── master
│       │   └── safia
│       │       └── my-new-branch
│       └── remotes
│           └── origin
│               └── HEAD
├── objects
│   ├── info
│   └── pack
│       ├── pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.idx
│       └── pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack
├── packed-refs
└── refs
    ├── heads
    │   ├── master
    │   └── safia
    │       └── my-new-branch
    ├── remotes
    │   └── origin
    │       └── HEAD
    └── tags

18 directories, 25 files

Now, let's run the git add command and observe what changed about our .git directory.

$ git add README.md
$ tree .git
.git
├── HEAD
├── branches
├── config
├── description
├── hooks
│   ├── applypatch-msg.sample
│   ├── commit-msg.sample
│   ├── post-update.sample
│   ├── pre-applypatch.sample
│   ├── pre-commit.sample
│   ├── pre-push.sample
│   ├── pre-rebase.sample
│   ├── pre-receive.sample
│   ├── prepare-commit-msg.sample
│   └── update.sample
├── index
├── info
│   └── exclude
├── logs
│   ├── HEAD
│   └── refs
│       ├── heads
│       │   ├── master
│       │   └── safia
│       │       └── my-new-branch
│       └── remotes
│           └── origin
│               └── HEAD
├── objects
│   ├── dc
│   │   └── ce97ef3d92d70d1385952ba7a9988908f3f23e
│   ├── info
│   └── pack
│       ├── pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.idx
│       └── pack-457ca7bffdeceb504dbf91cd58e3602f47a56ced.pack
├── packed-refs
└── refs
    ├── heads
    │   ├── master
    │   └── safia
    │       └── my-new-branch
    ├── remotes
    │   └── origin
    │       └── HEAD
    └── tags

19 directories, 26 files

Oh! Look at that! There's something new in our .git/objects directory. Let's take a look inside and see if we can find out more.

$ git cat-file -p dcce97ef3d92d70d1385952ba7a9988908f3f23e
# nteract <img src="https://cloud.githubusercontent.com/assets/836375/15271096/98e4c102-19fe-11e6-999a-a74ffe6e2000.gif" alt="nteract animated logo" height="80px" align="right" />

[![](https://img.shields.io/badge/version-latest-blue.svg)](https://github.com/nteract/nteract)
[![](https://img.shields.io/badge/version-stable-blue.svg)](https://github.com/nteract/nteract/releases)
[![codecov.io](https://codecov.io/github/nteract/nteract/coverage.svg?branch=master)](https://codecov.io/github/nteract/nteract?branch=mas