Field Guide To Design Finals

Though intense, you should approach the design event as something exciting.
You get to reach for the limits of all the things you've been thinking and talking about all year.
The whole competition is filled with people on the same wavelength.
It might even be fun.

TLDR - How to prepare for the design event:

Find someone to keep asking "Why?" until you don't know why you've always done it that way.
Until you have to go think about something for a week.
Do this constantly throughout the year.
Show your math, check your work, which ideally gets clearer and easier to explain over time.
If it never got further than a napkin, put the napkin in the binder.

Extreme personal opinion #1:

A finalist team should be able to walk through the entire design process.
From a napkin, to a whiteboard, to a spreadsheet, and only then turn to fancy computer simulations.
All the math beforehand, and the data from the running car afterward, are both essential protections.
Garbage in::garbage out wastes your time and embarrasses you.
You don't know a program until you can deliberately make it do something it's not supposed to.

Justin’s Extreme personal opinion #1:

A good finalist team can justify their decisions with first principles. They will also understand the tradeoffs and compromises made in their choices. Hand calculations and spreadsheets that explore the design space are much more valuable than an unverified FEA analysis of a rocker.

To all the new and inexperienced teams:

Just make the car go, make the car stop, make the car turn.
And practice passing the entire Technical Inspection process.
Along the way, document things that can make the car go, stop and turn better.
Design finalists do the same thing.
They write EVERYTHING down, then organize the information, then use it to manage the project.

Fix what can be fixed practically, show us what isn't working before we notice.
It proves you are paying attention, and shows you know how to fix it.
You are better than your car.
The best car I have ever built is the next one.

Never let a "better" idea stop the car from going, stopping, and turning at competition.
After that: Driver development is necessary for car development.
Both should lead to more dynamic points, and more importantly, more data.
We will be very disappointed if all the information gathered is not used to loop back through design decisions and understanding.

Extreme personal opinion #2:

It is not about the car at all.
It’s how well the team understands the car.
I only look at the car as a source of questions, and forget what it looks like as soon as it leaves the bay.

Fellow design judge:
"You have a beautiful car. But you did not convince me you could make an equally beautiful car from a blank sheet of paper."

In one finals, there was an incredibly simple and clean car right next to an incredibly intricate and complex car.
Both teams had incredibly thorough arguments and understanding of the details of their design choices.
The teams scored very closely to each other.
I had a very high opinion of both teams and both cars.

Design Presentation

Review Justin's design reports article.
As of 2022, the format is different - a slideshow - but the advice about content stays the same.

The chief judges read every presentation.
They use this, and experience, and the calendar of other static events to sort the teams into queues.
North American competitions typically have 8-16 queues.
Typically, they attempt to evenly divide the potential finalists, midfield teams, and fresh beginners.

Extreme personal opinion #3:

Teams should write design presentations from scratch every year.
Starting early in the year, and before reviewing old presentations.
You're not supposed to reinvent the wheel, but you learn a lot about wheels that way.
I will read all your previous year reports.
A team I watched climb from nowhere to finalist over a few years had wildly different reports every year.
I take repeat reports as a sign the team cannot go from first principles to the car, and is scared to change.

Fellow judge, late 2019:
Do you see this line in the design report? I wrote it. In 2008.

Main Design Event - Preliminaries

Non-finalist answer: "The test didn't work."
Finalist answer: "We tried to confirm this assumption with a measurement, but the test didn't work, likely as a result of this part of the setup. Here is all the data plus the things we were still able to learn."

It's not a presentation, it's a 30 to 55 minute dissertation defense.
The purpose is to rate the extent and cohesion of your team's understanding of your work and your car.
The most typical approach is to separate into Cockpit/Controls/Brakes, Frame/Body/Aero, Powertrain, and Suspension.
Review Chris Warren's field guide to the design event for how this may vary depending on the number of judges.
Assume that the judges have a reasonable understanding of the subsystem.
We prepare by going through the relevant sections of every presentation in our queue.

It is not like presentations to sponsors, new team members, or other non-experts.
For example: "This car has a double wishbone suspension" is not worth a page.
It should be three words leading to a much larger point.
For many judges, you could show the geometry for less than the blink of an eye.
We would already be asking what you think of those roll centers?
And what about the geometry during actual cornering?

Your design event material should try to anticipate as many questions as possible.
Prepare to delve into any topic on the design score sheet in terms of calculations, testing, and interpretation.
Some design review briefing slides might be a good starting point for a deeper discussion.
We are going to be in the appendices of your binders or electronic documents.
You should know them backwards and forwards and be ready to jump rapidly from insight to insight.

The closest the design event might come to a presentation would be if your judge chooses a long question, such as:
Show me the process of how you used tire data to make design decisions, especially from the previous car to this one.
Your goal is to communicate quickly, efficiently, and comprehensively.
The judge might budget 10 minutes to work through that question.
A good method is to make sure every point you make is leading to another point.
If I can see you understand a concept, I will hurry you on to the next idea.

I tend to start the conversation with a list of the topics I want to make sure we cover.
And ask about any topics you are especially proud of, and budget some time.
I ask tangential and interrupting questions.
It is perfectly reasonably for you to ask to quickly finish your current thought.
Or to suggest segueing to the next question via another topic.

Keep the information fast, dense, and nuanced.
Aim to share everything you've learned in your career in under an hour.

Extreme personal opinion #4:

Innovation (new invention) is not a thing in FSAE, or racing in general.
Functionally, after 140 years of automobiles and 40 years of FSAE, everything grows out of precedent.
It's largely iteration and applications of developments from real industries, historically aerospace.

This also means there are no “big performance secrets”.

There are still incredible levels of problem solving, which is a deeply creative process.
No one has ever built your car before, in all its particular solutions.
The best teams do the simple things very well, from a system level.

I really like this discussion: Component vs. configuration design.

Design Preliminary Scoring

After probing for strength and weakness in each section, judges assign raw scores.
If a team scores 20 or more in each of the big 25 point categories, they're almost definitely a finalist.
If a teams scores 20 or more in 3 of the main categories, and a 14 in the fourth, they're probably not going to finals.
A quarter of the finals judges will think they're missing elementary understanding.

Judges compile a long list from all queues.
Then there is a discussion among all judges to send a short list to finals.
The longer the list is at the start of the discussion, the easier it is to exclude a team based on a single weak area.
The target is generally around 10 cars.
That said, we want to give all true contenders a chance, and finals can have more than 10 cars.
If there are only a few strong teams, finals could have as few as three cars.

Raw scores of non finalist teams are normalized to round numbers.
"Letter grades" is a way to think about it.

Extreme personal opinion #5:

Some finalist teams are better organized and more cohesive than any company I've yet worked for.

Design Finals

In the main design event, your team needs to summarize a year's worth of useful information to 3-5 judges in a half-hour.
If your team makes it to finals, you will encounter 30-50 judges, each with their own graduate level questions.
Both reward efficiently communicating how everything works in context.
Your team also needs the ability to synthesize answers about new and unfamiliar questions.

We will also bring the largest and smallest persons at competition and ask you to make them comfortable in the car.

Usually, the process is for each judge to rank the finalist teams from first to last.
It's rare for any team to not have at least one podium vote.
It's normal for the overall winner to have multiple last place votes.

The rankings are averaged.
If they are spread out, the order is obvious, and we assign larger point differences.
If they are very close, we cluster point differences between places.

Example multi-part question, 10 minutes:
What are the factors that influence energy efficiency?
How does the energy consumption of your vehicle vary with speed?
How would you estimate the most efficient steady-state speed?
Accelerating from a stop in a straight line, how would you minimize the time over energy ratio?
What about for very short, medium, and long straights?

Your team may or may not be told how many parts of the question there are to work through.

In both preliminaries and finals, when you do not know the answer, start with "That's a good question."
If it is a hard question, this flatters the judge, and buys you time to think.
"I don't know" buys more time, then you should try to figure it out.
Do not make something up and call it a fact unless you want to score zero.
Asking for a clarification or re-phrasing of a question is also useful.
It can be appropriate to ask questions in response to the questions.
Can I assume X or Y? The answer may be "Of course, assume X and/or Y."
The answer may be "You tell me."
In that case, work through the implications of the assumptions.

For a while, one of my questions was:
Two motors with the same peak power.
One has a flat torque curve, one has a flat power curve.
Tell me how you would pick your gearing.

I could see all the blood drain from one student’s face.
He tried to think of all the additional information he would need.
By asking questions about each missing piece, over several minutes, he worked out the correct answers.
It was one of the best performances I’ve ever seen, and it earned his team a second place vote.

From the team that got my first place vote, the student said the drivetrain expert was talking to someone else, but maybe he could answer part of the question in the meantime?
Same question, he thought for 10 seconds, and then said:
”In the car with the flat torque curve, you would want the maximum number of gearshifts you can tolerate, to stay near peak power.”
”For the flat power curve, you would want the minimum number of gears necessary to cover the speed range of the car.”
His badge said “Suspension and Data Acquisition.”

Your team needs a deep bench to reach design finals and perform well.
One person who knows everything will leave judges with no one to talk to.
Every team member should be conversant on the basics of how different parts of the car interact with their system specialty.
The judges will subconsciously fill in small gaps in communication.
And they will call out important points that go unspoken.
Design finalists can easily communicate complex interactions in the same way they can teach their teammates.

Great teams work together to figure out difficult questions out on the fly.
Great teams seek out a judge to continue discussion during or after finals.

Question: What is the minimum factor of safety in the drivetrain?
Podium answer: Given the bumps at this venue, we calculated dropping the car 100mm while at full power.
Across these components, we increased the mass by 125g.
This increased the safety factor to 2 considering fatigue, and we never worried about them again.
All time great answer: Here are our calculations for every component from piston to tire while calculating a clutch drop.
Here are the weak points we identified and the corrective measures we took.
Would you like to see a brief presentation of our launch transient testing?

Extreme personal opinion #6:

Good performance in the design event leads to good performance on track more than the other way around.

After years of leveling up, a team made made it to finals.
Afterward they said, "When we didn't make it, you told us there's another level.”
”Now that we've been through that, we think there are at least two more levels."

Justin’s Extreme personal opinion #2:

The soft skills matter, and be proud about your work.

I had been asking teams about how they collect, handle, organize, and share data from their logger within their team. The data system team lead on a particular team showed me a printout he sent to the brake system lead, and told me “He asked for the brake pressure, but I knew he didn’t just want the pressures. So I also put the speed and longitudinal acceleration on the x-y graph.”

The data system lead not only showed me his technical competence, but also his understanding of other systems and ability to provide context in answering technical questions.

If I was looking to hire someone, I would have offered him a job on the spot.

Metaphor

Reality -or- Theory
-or- Design Target

Best available information.

All models are wrong.
Some models are useful.

Teams that do well in design recognize and explore the limitations of every tool.
Design finalists use insight, brute force, and everything in between to maximize the usefulness of their tools as it relates to the performance of their car.

Extreme personal opinion #7:

Any team can reach design finals if they spend enough time repeatedly and relentlessly ask “Why?” through the micro and macro, and document every answer.

I really like this discussion: Good design doesn’t cost money.