Took a few days off between Day 3 and Day 4 so just getting to write this post now. Turns out that what I wanted to do for Day 4 was actually simpler to do than I thought.
For my course, all assignments are in their own repo, but to avoid having many little repos, I put all the labs for the course into one repo and make them available within the first week. Students who wish to work ahead by reading the course notes are welcome to do so.
However as all labs are in the same repository, I do not want the submission of one lab to trigger a set of tests for every other lab. I was trying to figure out how to do this in the simplest way possible. It turns out, this is really simple to do. All I need to do is set up a separate workflow for each lab. For the labs, the starter files are all stored in separate folders. All that was necessary was to set up each work flow to trigger on changes made to files within a certain folder.
With this last piece, I think I am ready to use Github actions for testing my student’s submission. The next step is to go through my labs to ensure that the instructions are clear. This is not a short process but I believe the results will be well worth the effort. Not only will it make the assignment submission process be simplified, but it will also introduce students to CI process.
Yesterday I set up my first github action workflow that would build and compile my program. Today, I wanted to see how I can get github action to recognize when testing fails. The testers for my class are in C/C++. I had done a big overhaul to how I write those a few years ago. At the bottom of my testers I typically have the following if/else block
if(numPassed == numTests){
std::cout << "Congratulations! You have passed testing for A1 part 1" << std::endl;
}
else{
std::cout << "Looks like you still have some work left to do" << std::endl;
}
I have been trying to find some docs about this online but much of what is written uses testing frameworks and is usually not C/C++. I wanted to see if I could take what I have and make it work with minimal changes. One of the first things we teach in C/C++ is that the return 0; at the end of the main() indicates that everything went well. Any other return value indicates an error so I figured.. lets see what would happen if I simply returned something other than 0. Now, since my program works.. I figured the easiest way to test it was to put a return 1; on success in the tester (thus if it works.. the test would fail).
if(numPassed == numTests){
std::cout << "Congratulations! You have passed testing for A1 part 1" << std::endl;
return 1;
}
else{
std::cout << "Looks like you still have some work left to do" << std::endl;
}
I made this change to main, added the file and pushed to github. This is the result:
And yay! it failed. Lets see the details:
And there we go… an exit code of 1, is all you need from your main to indicate that it was not working properly. This is wonderful! The fix would be to simply add some return statements.
What about crashes?
I wanted to see how github action would handle an outright crash. To test this, I added some very likely to crash code… a simple array overrun… this should crash:
if(numPassed == numTests){
std::cout << "Congratulations! You have passed testing for A1 part 1" << std::endl;
int array[10];
for(int i=0;i<100000;i++){
array[i]=i;
}
return 0;
}
else{
std::cout << "Looks like you still have some work left to do" << std::endl;
return 1;
}
And sure enough.. it crashes and indicates a failure state.
This is amazing. So far GitHub actions is doing what I need it to do in an expected manner. I can see how actions will work.
Next Step:
Day 2 goal accomplished. For Day 3, I would like to change my workflow to pull the tester from a different repo. This way, if there is a flaw in the tester when it was initially released, a fix can be applied to a single source and the build will use the most up to date and correct tester. I already think I have a good idea of how to do this.. just need to try it out.
Cathy's Blog 