Tracking the original source of an edit

[sophie]

Sometimes, it's useful to be able to track how a change made its way into the codebase to start with, for tracking why something is a certain way when it seems totally nonsensical or non-obvious.

Fortunately, it's easy to do so with the versioning systems that Dreamwidth and LiveJournal use, which track every single change made to the source code. (It's quite literally possible to specify any arbitrary date and pull a complete copy of the source code as it was on that date.) In this post, I'm going to use this feature to track down the original source of the FIXME that foxfirefey mentioned in her wonderful bug walkthrough, to try to see if it's possible to find out what it's for.

First, I'll quote a bit from the original post:

I want to check and see how the message actually gets sent. I go back to cgi-bin/ljprotocol.pl to look at that function for clues:

494 my @msg;
495 BML::set_language('en'); # FIXME
496
497 foreach my $to (@to) {
498 my $tou = LJ::load_user($to);
499 return fail($err, 100, $to)
500 unless $tou;
501
502 my $msg = LJ::Message->new({
503 journalid => $u->userid,
504 otherid => $tou->userid,
505 subject => $subject_text,
506 body => $body_text,
507 parent_msgid => defined $req->{'parent'} ? $req->{'parent'} + 0 : undef,
508 userpic => $req->{'userpic'} || undef,
509 });
510
511 push @msg, $msg
512 if $msg->can_send(\@errors);
513 }
514 return fail($err, 203, join('; ', @errors))
515 if scalar @errors;
516
517 foreach my $msg (@msg) {
518 $msg->send(\@errors);
519 }
520
521 return { 'sent_count' => scalar @msg, 'msgid' => [ grep { $_ } map { $_->msgid } @msg ],
522 (@errors ? ('last_errors' => \@errors) : () ),
523 };

I wonder what that FIXME is there for. Put it on my list of followup items.

Let's examine where that FIXME comes from. To do this, we need to use a few new commands, the first of which is hg annotate. This command will print the file line-by-line, and next to each line, will give the revision number that last edited that line. Because it prints the whole file, we'll generally want to use grep with it to make sure we only get the line we want.

The hg commands only work in the cvs/ directories, so we need to know which repository to use first. As it happens, this file is in dw-free, so we change directory to that first, and then run the command with the file we want, piping it through grep to find the FIXME:

dh-sophie@hack:~$ cd $LJHOME/cvs/dw-free
dh-sophie@hack:~/dw/cvs/dw-free$ hg annotate cgi-bin/ljprotocol.pl | grep FIXME
  84:     BML::set_language('en'); # FIXME
   3:             # FIXME we ought to store this timezone and make use of it somehow.
   3:         # FIXME: this is a hopefully temporary hack which deletes tags from the entry
   3:         $limit = "LIMIT 200";  # FIXME: unhardcode this constant (also in ljviews.pl)
   3:             # FIXME: this goes away after we restructure APIs and
   3:     # FIXME: should be a utf-8 sort

It looks like that first line is the one we want, and it was last modified in revision 84. We now use another command to find the log message for that revision, called hg log. To use this, we pass it the -r switch, followed by the revision number of the log message we want to look at:

dh-sophie@hack:~/dw/cvs/dw-free$ hg log -r 84
changeset:   84:3f79143e7dd4
branch:      staging
parent:      82:1defc551af6c
user:        Mark Smith <mark@dreamwidth.org>
date:        Wed Jan 14 08:55:45 2009 +0000
summary:     First attempt at merging DW code with LJ r14766.

The 'summary' line is what actually contains the log message, and it looks like this code actually came from LJ, not DW. That means that in order to investigate further, we need to investigate the LJ repository instead.

The process is very similar, but because LJ uses Subversion instead of Mercurial for their versioning system, we need to use a program called svn instead of hg. Also, because we don't have the LJ code downloaded, we'll refer to it by a URL istead of a local filename.

The LJ code repository is stored at http://code.livejournal.org/svn/livejournal/trunk/ , which is their freely-available code repository. There are others, but we don't need to worry about those, because Dreamwidth can only use the code that's available in the freely-available repository - so if you need to go back to LJ code, this is the URL to use.

From here, we can start the process again with the new repository. However, because we already know from the log message that we took our code from revision 14766, we can restrict our command to start from that revision and ignore any changes that happened afterwards:

dh-sophie@hack:~/dw/cvs/dw-free$ svn annotate -r 14766 http://code.livejournal.org/svn/livejournal/trunk/cgi-bin/ljprotocol.pl | grep FIXME
 14119 ssafronova     BML::set_language('en'); # FIXME
  2758    martine             # FIXME we ought to store this timezone and make use of it somehow.
  5065  marksmith         # FIXME: this is a hopefully temporary hack which deletes tags from the entry
   857   bradfitz         $limit = "LIMIT 200";  # FIXME: unhardcode this constant (also in ljviews.pl)
  3010       avva             # FIXME: this goes away after we restructure APIs and
  2594   bradfitz     # FIXME: should be a utf-8 sort

Notice that svn's output shows you the username who committed that revision as well as the revision number itself, unlike hg. We could get this from the log too, but if all you want is the username, this makes it easier.

It looks it got changed at revision 14119. Let's look at that one. Again, we need to supply the URL to the repository:

dh-sophie@hack:~/dw/cvs/dw-free$ svn log -r 14119 http://code.livejournal.org/svn/livejournal/trunk/
------------------------------------------------------------------------
r14119 | ssafronova | 2008-08-05 14:36:18 +0000 (Tue, 05 Aug 2008) | 2 lines

LJSUP-2330: sendmessage command added, getinbox now return msgid & parent_msgid

------------------------------------------------------------------------

Hmm, that doesn't seem too informative. It might be clearer if we looked at what was actually changed here, by looking at a diff.

There are two ways we could do this. The first thing we could do is to run an "svn diff" command, which would show us a diff in the traditional patch format. To do this, we need to specify the two revisions we want to compare - in this case, we want to see what happened between 14118 and 14119, so we'd do it like this:

dh-sophie@hack:~/dw/cvs/dw-free$ svn diff -r 14118:14119 http://code.livejournal.org/svn/livejournal/trunk/

I've snipped the output of that command from here because it's rather long. In any case, there's a better way - using Trac, LJ's Web-based system that is much nicer, and available at http://code.livejournal.org/trac/livejournal/ .

On that page, find the text box labelled "View revision:", near the top-right. In here, enter '14119' and press RETURN. The page you get back won't be too much different from what you had before, except that the heading will read 'root @ 14119' instead of just 'root'. Once you're on the new page, select "Last Change". You'll be taken to a page showing the changes made in revision 14119. (note: This page may not be very accessible for screen readers)

Sometimes, examining the diff can show that what appears to be new code was actually moved from a different file, in which case we would then examine the history of that file instead. In this case, however, it looks like this is the origin of the FIXME; all the code is new, and the FIXME is still there in the middle, on line 367.

In a lot of cases, the log message for the origin of a piece of code will explain something about the code and what it's for. Other times, it's possible to look at the other changes made in the commit and figure it out from context. In this case, however, it looks like it's a FIXME that's been in place ever since the 'sendmessage' command was created, and hasn't been changed since.

The log message references "LJSUP-2330". This refers to a ticket in LJ's internal bug tracking system, which isn't open like our own Bugzilla installation is. Sadly, this is a dead-end too, as we can't see it.

So, although we weren't able to find out what the FIXME was for, we were at least able to find:

1. where it originated from (LJ's codebase)
2. when it was originally committed (5th August, 2008 - 11 months ago as of the time of writing)
3. who committed it (ssafronova)
4. and the ticket number for it in LJ's bug tracker (LJSUP-2330)

With this information, it might be possible to investigate more, if we had the resources. However, we don't, so this is the end of the road for us.

Hopefully this has been helpful to some people. :)

Comments

[foxfirefey]

Eeee this is sexy!

[azurelunatic]

Sometimes changelog is helpful, sometimes it is not.

http://community.livejournal.com/changelog/6554434.html

In this case, not really helpful.

[denise]

It's funnier as "svn blame". ;)

[vlion]

Interesting.

Have you done work with hg bisect? It seems that it might be a useful tool.

[sophie]

I personally haven't, though it seems like it it's intended for when you don't know what lines are causing a problem in your code. When you know exactly what line you're looking for, this way is best.

(and sorry for the late reply!)