Web Site Project Page

From GCD
Revision as of 00:29, 24 September 2015 by Handrews (talk | contribs) (Remove outdated info, move interesting stuff to "Tech Archive" page.)
Jump to navigation Jump to search

GCD Web Site Project Master Page

The GCD has recently moved to an entirely new server and code base! This page documents the ongoing work to implement more features and improve the new system, the initial deployment of which was somewhat rushed by the abrupt demise of our prior technical arrangement.

Our top priorities are:

  • Improving our ability to credit and document contributions
  • Improving the structure of our data for better display and search
  • Improving our user interface (*we need a web designer for this*)
  • Improving our search capabilities

We welcome assistance from programmers and non-programmers alike (yes, we have tasks for non-technical people!).

Development Environment

Figuring out what all that stuff in GitHub actually is, and which parts we're really using:

How to set up a development environment:

Please use the GCD Vagrant-based development environment

Please join the mailing list and email us if you would like to help out!

3rd-party Documentation

Bug/Issue Tracking and Workflow Management

New issues should be filed and tracked in GitHub.

Some issues are still filed as bugs in our custom Bugzilla instance. Bugs in the NEW state are available for anyone to work on, no matter who is listed in the assigned field. Bugs in the ASSIGNED state are being actively worked on, and you should comment in the bug if you have questions about the work or the expected delivery date. Please do not forget to accept your bugs into the ASSIGNED state before beginning work. Bugs are set to RESOLVED FIXED after they have been deployed to either the beta or production sites, depending on the branch involved.

TODO: Document the whole bug process more clearly.

Standards and Best Practices

Code submitted to the project should follow these standards. Review Board has been set up for code reviews, and all changes must go through code review and be approved before being checked in. Some of these guidelines are covered in places such as the Django documentation, but the ones restated below are items we feel are important enough to be called out here.

Using Review Board

NOTE: This is a bit out of date, as post_review has been replaced with rbt post. Our Review Board server is quiet old, but should work with either command line tool.

Most of what you need can be found in the Review Board User's Guide. Don't forget to install the RBTools package to get the rbt command line tool.

To Submit a Review Request

rbt post does send changes which have been committed locally as a diff to the server. You can use git revisions and the --parent option to send a diff for a set of commits diffed against something other than origin/master. Using local branches in git makes this easier if you want to post multiple reviews simultaneously.

Once you use rbt post to create a review, you'll want to fill out a few more fields before publishing the review to the list.

  • Edit fields by clicking on the pencil icons next to them. When you are done with the fields, hit enter [return] to save the field.
    • Fields that are multi-line text boxes will have an OK button.
    • Summary, Description, and Groups are required. You should not use the People field, but should instead always set Groups to "gcd" (without the quotes).
    • You should put the bug number (from the GCD Tech Bugzilla) in the Bugs field whenever there is a relevant bug. Even if the change does not completely fix the bug. [NOTE: We need to update our configuration to point to GitHub issues instead of Bugzilla]
    • Filling out Testing Done is strongly encouraged.
    • When proposing a significant change to the visual layout of the site, a screenshot is often useful. There is an "Attach Screenshot" link at the bottom right-hand corner of the panel for this purpose.

You can also click View Diff in the upper right-hand corner to make sure the changes being shown are the ones you intended.

Once you've filled out the fields, click Publish. You may need to click more than once if the system thought you were still editing one of the other fields (but wait and see if the first click worked for a bit- it should send email to the gcd-tech mailing list).

To Review Someone Else's Code

Click on the link in the email. Click View Diffs in the upper right-hand corner to look at the changes. Click on any line number to make notes about the code, and write your comments in the Comments tab, clicking Save Comment when you are done with the comments for that line. You will see your saved comments appear in the Reivew tab.

You may view existing comments (noted by a speech bubble next to a line number) by clicking that line number and looking in the Discussion tab.

When you have finished adding comments and responding to the discussion, click "Review" (which is a small link at the bottom right of the top panel of the review, above the diffs, or click any line number and go to the "Review" tab. If you want to send your comments without approving the code (because you are asking for changes or explanations), click Publish. If you approve of the review request, click Ship It.

(Note: "Ship it!" has long been the traditional approval response at VMware, from which this tool originates- one of the GCD tech contributors worked there years ago when this was all done by a Perl script. It is amusing to watch the Review Board authors defend this phrasing on mailing lists and bug reports instead of just changing it to "Approve" or something else more obvious to non-VMware people. It's important to note that the Ship It button does not check the code in. The developer who submitted the code must do that).

This review process repeats each time the author modifies the change in response to comments until the reviewer(s) are satisfied and approve the review. Unresolvable disputes may be appealed to the Head Programmer, or if the Head Programmer is one of the parties, to the Technical Coordinator.

To Make Changes to a Review Request

If you are asked to make changes to a review you submitted (and agree to make the changes), then after you have finished your changes, re-run post-review, but with the -r argument giving the existing review number for your request. This will cause the tool to update the diff of your review instead of creating a new one. For instance, following the original example (and assuming no new files were added to the diff, or old ones deleted from it), and assuming the review is number 42, the command would be:

    rbt post -r 42

Or if you used --parent when originally posting your review, you should pass the same value for parent again:

    rbt post -r 42 --parent=foo

(Note that the script should remember your server, so you probably won't need --server all the time unless you're using review-board somewhere else as well)

Once you've done that, you once again go to the review, make any additional comments or respond to discussions in the same way you would if you were reviewing the code yourself, and then click Publish again. You may also make comments and change fields without submitting a new diff. Be sure to hit return if you are changing any of the fields to save your edits before you click Publish.

Committing the Code

Once a reviewer has approved the change, you should push to GitHub. Please check with the gcd-tech mailing list about using a fork and pull request vs having direct push access. We have not really settled on a policy yet.

After that, you should go back to Review Board, go to the review screen, and click the "Set Submitted" link in the bottom right-hand corner of the main review panel (do not click View Diffs first, as the Set Submitted link does not appear on that page). This will remove the review request from the active list of reviews in everyone's dashboard.

Abandoning a Change

If, based on feedback or your own reconsiderations, you decide not to check in a change, then instead of checking in and clicking "Set Submitted", go to the same location and click "Discard Review". This also removes the request from the list of active reviews.

Standards For Python Code

Write Python 2.7 compatible code.
This is our current production environment. Python 3 is not yet supported.
Follow the Python Style Guide. Read the whole thing, as it is full of useful advice.
This saves us from arguing over which of our personal styles should be used. Code that does not meet these guidelines will not be allowed through code review. If you find existing code that does not comply, please consider cleaning it up. Do not consider it an excuse to write more non-compliant code.
Read Python Idioms and Anti-Idioms
And do what it says :-)
Read the Docstring Guide
And do what it says :-)
Always place one space after each comma separating function arguments.
Unless of course there's a newline. For some reason this is not mentioned in the style guide, although all examples there follow it.
When breaking a statement across two lines, if there is no obvious way to indent it, indent two characters more than the start of the statement.
This makes it clear that the continued statement is not another indented block. When possible, line up continued statements by lining up arguments vertically, or the continued operand in an expression with the operand on the previous line. But when no such rule applies, use two extra spaces. Examples:

Breaking with an "obvious" indentations scheme of lining up operands vertically:

    foo = (bar.some_numeric_thingy +

Breaking when there is no obvious indentations scheme (so two spaces are used):


Note that it's often better to just break things up into multiple statements by using a few local varaibles in these situations:

    thingy = SomeClass.some_list_of_things[indexy_bit]
Do not use global variables. Ever.
If for some reason you feel you must do this, you must get approval from the Head Programmer first. The current acting Head Programmer is unlikely to approve such a thing. If 3rd-party libraries or frameworks rely on global variables, you may of course use them as required.
Make everything portable, and especially use os.path when working with files and directories.
The project currently has only three active developers and a shared test area, but already needs to run on MacOS, Windows and Linux. We also do not yet know what our hosting environment will be. So don't do anything that will limit us.
When capturing from regular expressions, use named capture groups, not positional groups.
This make regular expressions more robust if the string ever changes and capture groups get reordered. Additionally, while the regexp itself is a bit more cluttered, the intention of the capture groups is much more clear. Examples:


    m = re.search(r'foo:\s*(\w*), (\w*)', target)
    foo = m.groups()[1]
    bar = m.groups()[2]


    m = re.search(r'foo:\s*(?P<foo>\w*), (?P<bar>\w*)', target)
    foo = m.group('foo')
    bar = m.group('bar')

For Perl Code

Where possible, be compatible with the Python standards listed above.
Exceptions are obvious language community differences such as packages being CamelCase in Perl while they are lower_case in Python. Use Perl-style package names for Perl packages. Other exceptions should be documented here as they are noted.

For Django Models

Do not refer to Views, Templates or site URLs.
Models are pure data representation. They should present what is in the database, and sometimes expose methods that make that data easier to work with. But they should essentially be unaware of the higher layers of the application. Note that model fields that contain URLs because the database contains URLs are OK. These are generally external links anyway (like the URL field for publishers).
There is one notable exception to this rule, which is the get_absolute_url() method which is a Django convention for models. The Head Programmer does not like this method, because it ties the model representation to a single URL when we may wish to have several different URLs (for instance, one for display purposes, and one for REST API purposes). However, given that parts of Django expect this convention, it may be implemented and the basic display URL for the model should be used.

For Django Views

Use named groups in the URL conf for argument passing.
This makes view invocation more robust.
Use named URLs to disambiguate reverse mappings.
This supports better abstraction when code needs to look up URLs.

For Django Templates

We are still developing our template style
Suggestions appreciated. For now, stick with the following rules, and when you're working on a template that doesn't already conform, please try to leave it in better shape than you found it.
Template tags should have exactly one space inside of the enclosing punctuation
Example: {% if bar %} <span class="data">{{ foo }}</span> {% endif %}
Use two-space indentations
It's easy to get a lot of indentations in these things, so let's not get carried away.
Indent HTML and Django template tags independently
Trying to keep up with one indentation scheme results in weird misalignments when you just look at HTML or just look at template tags. So don't try. Indent all of your template tags consistently with each other. Indent all of your HTML tags consistently with each other. Don't worry if they don't make indentation sense together.
Please try to keep content within 80 columns, but do not go through confusing contortions in the process.
In other words, if you've got long URLs or if you need to string things together due to the vagaries of how Django's template engine performs substitutions, go ahead and let the line be as long as it needs to be. Sometimes, due to how Django templates treat space, this can be very long indeed.


Use 2-character indentation for each nesting level.


All CSS goes in external style sheets.
When modifying styles dynamically, it is better to modify the element's class(es) than set styles directly.
Classes and IDs are always lower_case_separated_by_underscores.
Colors are always associated with class names that define an elements role in the color scheme.
i.e. background, foreground, hilight, etc. Actual list TBD.

For JavaScript

We have not yet determined how and to what extent we want to use JavaScript. Please discuss any ideas with the Head Programmer first. Input on this topic is welcome.

Historical Note

Some interesting historical info that used to be on this page but became outdated enough to be confusing has been moved to Tech Archive.