A Faculty Advisor’s Perspective on Formula Design Judging
A couple of this site’s frequent contributors asked me for an article about FSAE/Formula Student Design Event judging from a faculty advisor’s point of view. Although I am delighted to do so, please note that advisors are as varied a bunch as Design judges, and so this view is only a little more comprehensive than my personal view. As background, I was Auburn’s advisor from 2001 to 2021, was just as involved with our off-road cousins, and have recently served as a Design judge in Michigan and overseas.
Advisors differ by: level of commitment (from totally absorbed to barely present); personality (from self-effacing to self-aggrandizing); expertise (from acknowledged expert to mere enthusiast); the expectations and pressure applied to them (from relative freedom to stresses similar to those placed on an SEC football coach). The advisor’s perspective will change according to their own background, character, and situation.
Recall that the purpose of an advisor should be: team memory; set standards; sounding board; encouragement; facilitator; institutional liaison (and shield); flushing the students out of the truck stop snack aisles on the road trip to competition; and serving in loco parentis when that trip goes badly. The advisor must never (i.e. never), per Rules AD.5.1.3.b & c, do any of the team’s work for it. So an advisor at Design judging may vary from totally detached, to the best of the above, to so in-your-face that they need to be chased away.
Advisors all care about whether their team gets its due. Some understand the relative merits of the other teams and can see their team in context – to them, feedback is almost as important as place. Others don’t see much further than wanting to know why their team didn’t win.
Advisors look at points and place and who made finals, sometimes wondering ‘why them and not us’. Advisors also listen to the comments of the students right after judging. Sometimes the students think Design went ‘fine’ (though good Design judges push the team, and wise advisors understand that the team’s objective in Design should be to score its best, and not to come away feeling comfortable – to paraphrase Carroll Smith on race drivers). Sometimes the students feel that it did not go ‘fine’ and might present an excuse instead of owning up (‘the judge was prejudiced against me in the following way’). Sometimes the students think the judges are seeking a very particular solution to the open-ended question of design of a formula car and its supporting analysis, and are not much interested in what the team did to solve the same problem by alternate means and under different constraints. (I hear this a lot; understandably, pursuing commonality of process makes it easier to compare one team with another, but in doing so, the open-ended nature of both Formula and engineering design is ignored). Occasionally the students feel that a judge is unfamiliar with Formula and asks questions and expects answers that are inappropriate for the series. But mostly, the students feel that they had a really good talk with a knowledgeable person who fairly discussed the virtues and shortcomings of their design process, as illustrated and proved by the result of the car on the ground.
An interesting observation is that sometimes a team’s area scores are questionable – both too high and too low. But the high and low tend to balance in the total score to something that seems about right. For instance, I’ve seen challengeably low structure scores balanced by don’t-remind-the-judge-of-the-mistake-they-made powertrain scores and resulting in a total score that was, overall, what I thought the team deserved. Colleague advisors have observed the same thing. Of course, one cannot depend upon these outlier area scores to always offset correctly. But I would say that when skewed area scores do balance, it results from a panel of judges who all consider the car as a whole, in addition to focusing on their specific area.
Suggestions for Design judges based on my experience as an advisor:
Don’t over-focus on your assigned area. Judge your area as an integral part of the whole car. Whole car design is what we are after, after all.
Focus on how the team’s design process resulted in their design conclusions – as opposed to focusing simply on the design outcome. Design outcomes that might seem odd, given your experience with another series, might be right in this series, for this team, and the constraints that confine its design space.
Shield yourself against gullibility. As a professional engineering educator, having experienced countless students performing in testing environments (like Design judging), I can promise you that they will say anything that might plausibly lead to a higher grade (score). As professional engineering students, and FSAE/FS at that, these students follow this practice very, very well. And don’t tell me that in your student career on a team you didn’t also create fluff, embodied as analysis, plans, or even stuff you put it on the car, just to get Design points (hopefully, Design caught you and discouraged you). That is why the car on the ground is the truth, and not the stories.
Design should be necessary & sufficient. The definition of the sum of the process is the car on the ground; the proof of the process is the lean engineering case that the team makes for why, working under their particular constraints, out of the countless solutions that could have been selected, the car on the ground is their correct answer to the formula design problem. So long as the logic fully supports the car (and its testing data), and no logic is presented that does not wind up impacting the car, the engineering design case is lean (and therefore proper).
Does design analysis support design? Say a team has beautifully presented STAR-CCM+ sweeps over ground clearance, roll, yaw, and even pitch for the full aero package with good modeling of clutter, HX, fan, vortices, and separations. Do they deserve any Design points if they cannot establish that all this information led to optimization of the aero package, or to optimization of the suspension that resulted from lap simulation with accurate point-by-point aero inputs? I.e., will their analytical ability and understanding result in more dynamic points? Have they used the theoretical to generate the practical? I think that we are doing both the students and judges a favor if we insist upon lean design.
Sometimes, in a team’s effort for Design, feedback is valued over score. A lot of teams blow off Design – they show up to get what points they can, and hope for good feedback that will make them better next year. A lot of teams build their presentation for Design after their car design has already been handed over for fabrication, and so their claims of design process are sort of fake. I have been associated with a few teams of each type. This conduct is not necessarily bad.
One might conceive that there are two major learning outcomes in Formula. The first might be called the build-a-car-and-make-it-work learning outcome. In an engineering education setting where very few projects are seen through from vision to a running reality, this might be Formula’s most valuable learning outcome. Teams going more or less exclusively for this outcome just want to get into the Formula game. They do Design to benefit from the expert design feedback. With good feedback, these teams might seriously attempt the second learning outcome in future years.
The second learning outcome might be called excellence-in-engineering-understanding-leading-to-excellent-car-design. I think that the Design community has established that good design correlates strongly with good dynamic performance, and that dynamic performance by cars without good design (by somebody, sometime, somewhere) is rarely good. The second learning outcome leads to Formula success.
I think that the judging mission in Design is to take that first outcome (which the advisor usually has a hand in helping the team see how to achieve) and, by setting a standard and giving expert advice, propel teams up the mountain of the second learning outcome.
My conclusion (based on my perspective, and not necessarily on the points above) is that FSAE/FS Design judging is the world’s best in engineering student project evaluation. The students care, the judges care, the expertise and relevant experience is there, and the Design judging process, hammered out by compromise from a wide variety of well-informed views, is highly respected. There is no other engineering student design competition that I can say the same for, and I have experienced many.
Getting personal, I have always felt that my team got about what it deserved from FSAE Design judging. I have never felt that my team was shortchanged, or alternately got away with pulling a fast one. As to how seriously Design is taken, on a cusp-of-Design-finals team such as mine, I can promise you that the hope for improvement in just that 15% of total FSAE points absolutely drove the next year’s design process. And, except for the fluff generated in that pursuit (call it a waste fraction, if you like), I think that this is a good thing.