Branches And Development Process
Weblocks development process occurs in three Mercurial branches - development, stable, and experimental. All branches are hosted by http://bitbucket.org. New features are normally pushed to the development branch. The features are tested, improved, and eventually merged into the stable branch. The experimental branch is a Mercurial patch queue based on the development branch. Experimental features are intended to be pushed into this branch as Mercurial queue patches. These patches may later be dropped, or merged into the development branch.
If you introduce new features, please provide unit tests. If you fix an untested bug, test for that as well. We're happy to merge features we won't be using ourselves if it's made reasonably robust via testing; otherwise, we're likely to break it with other changes.
A corollary is don't break tests. Getting all the tests to succeed again after the big post-Darcs merge was hard enough; no one wants to let it drift out of control again, and we'd rather let features lie than break a bunch of tests again. If you break a test and think the test itself should change or be removed, think twice before doing so, because if the complexity of test changes outweigh the source-side benefits of a change, we won't merge it.
Upon introduction of features that change CSS classes, introduce new HTML containers or remove old ones you also need to make sure that the default CSS rules still apply. A rough way of checking this is running the demo applications. If the CSS needs to be changed, include it in your patch.
The demo applications must be checked to still work after your changes. If they don't then please include the appropriate changes to make them work again in your patch, too.
The stable branch can be browsed at http://bitbucket.org/skypher/weblocks-stable/, and cloned by running the command below:
hg clone http://bitbucket.org/skypher/weblocks-stable/
This branch contains well tested code that is suitable for running a production web site. Normally, large bug fixes and new features are merged into the stable branch only after it is extensively tested in the development branch.
The development branch can be browsed at http://bitbucket.org/S11001001/weblocks-dev/, and cloned by running the command below:
hg clone http://bitbucket.org/S11001001/weblocks-dev/
This branch contains code that may contain bugs, requires more work, or causes problems with the test suite or other features in the repository. After code is perfected and tested in the development branch it is merged into the stable branch.
Developing and maintaining Weblocks is fun, but it takes a lot of time. We very much appreciate people submitting patches and we'd like to accept as many useful patches as possible, but unfortunately some submissions require us to carve out chunks of time that we do not have. To make the maintenance process easier on the core team and to increase the likelihood of getting your patch accepted, please follow the guidelines below:
- Add a documentation string for every function/class/slot/parameter you're adding. If you're modifying something, please be sure to also modify relevant documentation strings.
- Unit test every feature you're adding. If you feel something you've added isn't possible to unit test in a useful manner, let me know about it.
- Follow the overall code and documentation style of Weblocks codebase. I know it's not perfect, and one day I may write scripts to change it, but consistency is more valuable than tiny bits of styling perfection.
- Integrate your changes into Weblocks codebase.
- If you're adding new files, also add them to relevant ASDF files.
- Put files in the appropriate location (parts of weblocks under src, unit tests under test, scripts under scripts, publicly downloadable files under pub, etc.) If you're not sure where to put something, ask me.
- If you're adding new unit tests, make sure they don't affect the testing environment in a way that may break other tests.
- Make sure existing unit tests pass.
Getting along with Mercurial, Bitbucket, and Weblocks
The tools and services we use, and the surprising number of public Weblocks branches, can be somewhat daunting. However, with a little reading, and by watching other developers' feedback, you can avoid unnecessary work and provide optimal contributions for rapid inclusion into mainline.
Never rollback a change you've already pushed. If you do that, you'll see a message about creating new heads when pushing again, and things will get a little confusing. Instead, use backout (followed by merge and ci if necessary) to remove past changes that you've already pushed. If you accidentally rolled back, just pull from the Bitbucket side until you can get a single head going again.
Bitbucket forks do not automatically track their parents. For example, if you fork dev, and we push new changes to dev, you have to pull from dev, maybe merge, and push for them to show up in your repository. This is especially important if your intention is to contribute changes for dev.
I just want to try contributing one or two small patches…
To submit the patches please publish your repository clone on http://bitbucket.org (or elsewhere) and initiate a pull request on the appropriate branch's Bitbucket page. To publish on Bitbucket, fork the appropriate Weblocks branch, clone the fork locally, and make all your changes in that clone. Initiate the pull request for the appropriate repository (see branch descriptions above). If you're not sure which branch should be used, please ask on the weblocks group. Detailed Bitbucket steps follow.
- Log in to bitbucket.org
- Go to the appropriate repository, normally http://bitbucket.org/S11001001/weblocks-dev/
- Click the "fork" button. Make sure your fork is not a private repository.
- Install Mercurial on a machine somewhere; I'll assume you're using the command line interface.
- Go to your fork in bitbucket; get the Clone URL
- Run: hg clone [Clone URL]
- Make your edits for the patch you want to submit
- Run: hg commit
- Run: hg push
- Wait a bit, then check your forked repo on bitbucket for the changes
- Go back to the main repo on bitbucket.
- Click the "Pull Request" button.
- Select your forked repo in the dropdown
- Explain your changes in the text box
- Check all the boxes beside people's names; by default it sends to no-one.
- Click send request
- Leave the branch in place until the devs say they are done with it
I want to develop a new feature on stable.
Features are never pushed directly to stable without a stint in dev, so you should read the next section on working behind dev. If your feature can't reasonably merge into dev, it likely won't be merged at all.
I want to develop a new feature without continually merging dev.
That's fine, but before we will accept, you need to do a trial merge. I (S11) keep a pristine dev repository locally for just this purpose. Pull and merge the latest dev in, fix conflicts, and test it all again to make sure it still works. If the conflicts aren't trivial, you may want to push the merge anyway, just so we don't make a mistake when fixing them. If a test breaks after merging dev because dev "tests harder", you can put the fix in before the merge instead of pushing the merge with the fix on top if you like, because our intention in such cases is usually to say "Weblocks should always behave like this".
A good reason for conflicts in trial merges is that we've already pushed something similar to dev. So if your change is small, make sure someone hasn't already done the work you're doing in dev itself.
You might consider using Clozure for speed when doing trial merges if you are using SBCL or CMUCL. Clozure has a much faster compiler and fasl-loader, which can be helpful if you are frequently clearing out stale fasls and rebuilding from scratch.
I want to track dev while working on my changes.
The best thing to do is regularly pull from dev, merge, test, and keep working on the branch.
But now you've heard that there's a way to have your topic branch track dev without having the merge revisions appear in your log. It's called rebase, and its use is rife with problems.
An ASCII-art representation of how rebase works can be seen in the git-rebase(1) man page. A more colorful one can be seen on S11's blog.
If you use it, you'll be seriously impeding or breaking:
- Pushing your branch to Bitbucket. Pushing only creates new revs, and rebase implies "stripping" the old version of each rebased rev. But see Bitbucket meta #402 for the somewhat evil change request to fix this. A little analysis of Bitbucket's features will show why this is not a nice feature.
- Sharing incomplete features, accepting feedback and hacks from others. Cross-merging makes sure everyone involved is working from the same base, and reliably records when different ideas were implemented in revs for the feature and when changes from multiple hackers were put together . Rebasing wipes out much of this information, and more seriously, causes the "cascading rebase" problem detailed in the same blog entry mentioned above.
So the existence of rebase encourages you to consider that "clean history", whatever that means, might be more important than the above things. But these things are really good. Being able to share a contribution to someone else's topic branch, without worrying that you might have to later manually analyze the whole topic history again to complete a cascade, is good. Letting other people provide ideas for a topic before fully investing in a complete, ready-for-dev topic branch is good. Is it really terribly important to not have to pass -M to hg log, or not to wait for Bitbucket to implement "don't show merge revs" and "don't show revs in fork parent"?
Even hackers who take the time to implement a "rebase" function think it's a bad idea pretty often. So take a minute to consider whether it is really a good idea for you. From the git-rebase(1) man page:
Rebasing (or any other form of rewriting) a branch that others have based work on is a bad idea: anyone downstream of it is forced to manually fix their history. This section explains how to do the fix from the downstream's point of view. The real fix, however, would be to avoid rebasing the upstream in the first place.
If you want to do it anyway, here you go.
$ ls dev/ my-topic/ $ (cd dev && hg pull -u) # update "pristine" local repo pulling from http://bitbucket.org/S11001001/weblocks-dev/ searching for changes adding changesets adding manifests adding file changes added 2 changesets with 4 changes to 3 files $ cd my-topic $ hg pull ../dev # pull in new dev changes pulling from ../dev searching for changes adding changesets adding manifests adding file changes added 2 changesets with 3 changes to 2 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) $ hg heads # Note the higher number (tip) is the just-pulled revs changeset: 1176:56a0f397355b tag: tip user: Stephen Compall <email@example.com> date: Sun Jan 11 17:21:21 2009 -0600 summary: Build date printing around metatilities:format-date instead, testing # And here is the alternate head created by your topic changes changeset: 1174:cd08a2361295 user: Stephen Compall <firstname.lastname@example.org> date: Fri Jan 16 22:50:50 2009 -0600 summary: my-topic: some other change I made $ hg rebase -d tip # do the rebase (only works when `hg id` is cd08a23) merging weblocks.asd merging weblocks-test.asd saving bundle to /home/sirian/lisp/weblocks/my-topic/.hg/strip-backup/d72aa32aabad-temp adding branch adding changesets adding manifests adding file changes added 4 changesets with 6 changes to 4 files # 2 added + 2 in topic rebase completed $ hg heads changeset: 1176:00618ea0af7e tag: tip user: Stephen Compall <email@example.com> date: Fri Jan 16 22:50:50 2009 -0600 summary: my-topic: some other change I made
I have a bunch of unrelated changes in my fork, but you tend to cherry them.
This is a good reason to watch how we tend to adopt your changes. Different people have different approaches to hacking Weblocks, and this encourages us to think differently about how to pull in changes from each person.
You can help us, and yourself by adapting your workflow to how we pull in your changes.
If we pull in your stuff wholesale, you might as well just keep committing to your repo, tracking dev or stable as necessary. There's no benefit to the more complex systems if we don't cherry from you.
If we're cherrying alot from you, that makes hg out mostly useless for figuring out what divergent changes you still carry around. In that case, you should try one of the patch management systems for Mercurial. If you tend to make each change in a single revision, Mercurial Queues are what you want. If you tend to work on different topics in multiple revisions, sometimes overlapping, try patch branches. With either of these systems, watch us for what we pull in; that way, you can remove these patches from your repository when appropriate, and the remaining list of patches reflects what divergences you still carry.
You merged my changes, now what?
Observe our commentary. Often, a push of a merge of your changes will also include the merger's changes. You can consider this a commentary on your changes described in software. We rarely announce such things because merge-with-commentary is such a common thing in free software projects, so you should watch dev after your changes are merged in for their appearance. Just because they don't appear immediately doesn't mean there won't be commentary; a mainline developer other than the merger might have something to say.
If you disagree with a commentary so much that you want it changed back you have to tell us. Further changes of your own that don't include the commentary will simply be treated as ordinary conflicts and fixed as if the commentary's inclusion is assumed. This is because your disagreement isn't reflected in an actual Mercurial revision; it is just a divergence. If you disagree with a commentary, but only that it should be changed or enhanced, the appropriate thing is to pull in the commentary and make further changes on top of it. This way, the revision graph reflects the actual conversation behind the changes.
Now for the technical side: get the hgk extension, because it gives you useful revision graphing. Some Weblocks developers arrange commits so that commentaries appear as direct children of the revisions they comment upon. Sometimes. When they feel up to it. Here's an interesting case of a dual parallel commentary appearing in an unusually complex web of merges. Different connections are in different colors for contrast, and the colors of text labels indicate different authors:
Ignore the web, focus on the fuschia line. No, not the one on the left that goes up into oblivion. Start at added template-block-mixin. Okay, ignore the little one that goes to the right, that's just another change from you. Focus on the other things it leads to:
Remember that further commentaries may follow on a single line; this was just an instance where they were made separately.
Here's another, longer example brought on by the interesting graphing algorithm:
Hint: it's the big brown one on the right.
This is a backout, meaning that the commentary means to say that the change should be reverted entirely. If you have dev, find this revision and look at the merge that follows it for an explanation of why it was backed out.
Anyway, using strict DAGs for revision graphing means it's kind of a pain to use direct-child commentary for non-trivial changes, so be sure to look down the log for other changes that might be related, implicit commentary.