Since the launch of the Codecademy course creator tool, we’ve been so lucky to have such smart and dedicated people writing and maintaining courses for our platform. In that short period, in addition to our own experience with the tool, we have received a lot of feedback from course creators about the major pain points of creating and testing a course. So lately we have been iterating and shipping small features and many fixes to the course creator. However, there is one point we haven’t really addressed — how hard it is to check the correctness of the student submitted code, especially in the "Web" courses.
Course creators have to write tests, Submission Correctness Tests (SCT), that would run against the students’ submitted HTML, CSS and JavaScript after they’ve been processed and rendered in an in-browser iframe. The only convenience we gave our beloved creators was the fact that we loaded jQuery for them. However, they had to pretty much do all of the heavy lifting.
The old submission correctness test is wrapped with a function that, when called, expects one of the following return values:
- true: Test has passed.
- false: Test has failed with an unknown error.
- Error string: Test has failed with an error message to show to the
student.
Example from one of our recent Web courses:
if ($('div').length === 0) {
return 'Did you add a new div to your page?';
}
if($('div.post').length === 0) {
return 'Make sure your new div has a class name of "post"!';
}
if ($('div.post > h2').length === 0 ||
$('div.post > p').length !== 2) {
return "Your div should surround the h2 and two p's";
}
// Regexp to match color in hex and rgb.
var rColor = /ccc|rgbs*(204,s*204,s*204s*)/i;
if ($('div.post').css('border-bottom-style').match(/dashed/) == null ||
$('div.post').css('border-left-style').match(/dashed/) == null ||
$('div.post').css('border-left-color').match(rColor) == null) {
return "Make sure you give your post div a border.";
}
if($('#nav li').css('padding-top') !== '5px' &&
$('#nav li').css('padding-left') !== '5px') {
return "Give your nav elements proper padding!";
}
return true;
Here are the major pain points and what we can do to make them better:
-
Expressiveness: An SCT is very verbose and there are repetitions in the code.
-
Browser differences and incompatibilities: Even though jQuery greatly helps normalize browser differences, there are many things that it just doesn’t help with. For example, when trying to get an element’s color using
$().css('color')
you may get one of the following three formats:- The rgb code of the color.
- The hex code of the color.
- The English name of the color (if you originally specified it using the name).
Another noticeable issue in the previous example is that checking the border or the padding, using the shorthand notation, isn’t possible because most browsers would return an empty string if you try to get
$().css('border')
or$().css('padding')
. In order to check those CSS rules, you would have to specify and check each direction and style. -
Readability: An SCT is hard to read, understand, and maintain across all of the browsers that we support.
Enter jquery.expect
jquery.expect or $expect is a simple DOM assertion library built on top of jQuery and is inspired by LearnBoost’s excellent expect.js library.
Without further ado let’s rewrite the previous example in $expect:
$expect('div').to.exist('Did you add a new div to your page?');
var missingPorHMsg = "Your div should surround the h2 and two p's";
$expect('div.post').to.exist('Make sure your new div has a class name of "post"!')
.and.to.have.children('h2', missingPorHMsg)
.and.to.have.children('p', missingPorHMsg)
.that.has.elements(4, missingPorHMsg).end()
.and.to.have.css( 'border'
, '1px dashed #ccc'
, "Make sure you give your post div a border!");
$expect('#nav li').css('padding', '5px', "Give your nav elements proper padding!");
Thus, we have transformed a very verbose and unreadable test into a succinct and elegant one. $expect is very fluent and expressive and we think it would immensely help in writing SCTs and assertions in general.
$expect has many other features that you can find in the documentation. One new feature is event testing, which was impossible with the old framework.
$expect('button').to.click(function () {
$expect('div').to.be.hidden();
});
$('button').click();
The new $expect library is well tested and stable. Nonetheless, jquery.expect is still in the early development stages and your feedback is greatly appreciated.
We think other people and projects may benefit from a library like jquery.expect, so we have made it available on Codecademy’s github under the MIT license. Happy testing!