Forem: Brian Kirkpatrick

[23/52] Game Engine-ering: Railway Empire DLCs Ranked

Brian Kirkpatrick — Sat, 23 Nov 2024 07:06:03 +0000

We're going to introduce a new sub-series this week. If you haven't noticed in our previous "software development" projects, I'm a big fan of game design and engine analysis. We're going to apply some of these ideas in our consideration (and ranking) of different DLCs for the great game "Railway Empire". Get ready for a tier list!

Metrics

We're going to identify several key mechanics for the core game and across scenarios to extract specific metrics (more qualitative than not!). We'll consider these metrics for each DLC and rank them on a S/A/B/C/F grade. Of course, these metrics are heavily weighted by how closely related they are to the parts of the game I enjoy! So, your mileage may vary.

At its core, "Railway Empire" can be described as a non-Cartesian Factorio. It's a basic logistics management game, with some degree of operations scheduling (I think of it as a fluids transport problem!). The non-linearity of the networks you build, as well as the relationships between different goods you transport and produce, mean that geography in particular is a key consideration. Building lots of tunnels and bridges, with no thought for terrain, will be exceedingly expensive. There are also considerations for predicting and planning how resources will be gathered and distributed, as well as the usual "economics game" mechanics like investment, buyout, and research.

But one thing that makes scenarios particularly interesting in Railway Empire is the degree to which they treat historical context with a lot of love. In the core game, this means real considerations for issues of technology development, trade patterns, transcontinental rail routes, and the critical difficulty of crossing the Mississippi. In various DLCs, this means fun insights into contemporary technology & politics, as well as entertaining lessons in geography and historical events.

So, our basic metrics for consideration are:

Topography: Is it balanced & realistic, well-utilized over the whole of the map, varied (but not too asymmetric), and a good mix of challenge and feasibility?
Scenario: Are there unique mechanics and decisions in the scenario? Are the objectives challenging? Does the scenario contribute to replay-ability?
Learning: Are there interesting insights into history and geography? Do you learn about interesting real-world events, trade patterns, and historical characters?
Artwork: Are different models and other artwork customized for that particular scenario? How many custom entities (like engines, player avatars, resource trees, and landscape features) are interesting and unique to that scenario?

With that outlined, let's get going.

Argentina (South America)

This expansion actually includes three scenarios (Argentina, Brazil, and Peru), even though they are each the same map and scenario from three different perspectives. But it does give you three perspectives of development in South America, with significant impact on patterns of growth and impacts of geography.

The map is a healthy variety of geography that covers a decent amount of the continent. You can appreciate how complicated a continent South America is, and how challenging it would have been to develop rail networks across different countries.

Good things about this DLC include the fact that there are multiple scenarios. The characters and personalities you encounter are interesting and entertaining, and there's a great variety of geography. There's a decent spread of cities (though with one void in the middle that forces a specific predictable pattern of north-south arteries).

But the geography is not particularly well balanced or distributed (good luck if you play a competitive match starting out in Chile!). Resources aren't particularly unique.

As a result, I dropped this one in "B" tier.

Australia

The Australia background menu art is great--something I actually hadn't originally considered. But the best thing about this DLC's scenario is the historical perspective. You are shepherding the growth of an entire continent from it's origins as a penal colony all the way to robust and industrialized independence.

The custom mechanism in this DLC (a "settlement" mode that lets you customize the manner in which new cities are established and developed, and in turn lets you "unlock" different resource sites) is interesting, and while it can be a little annoying it's nice to have a greater degree of customization over how the map develops over the course of a game. But there's a real sense of history as those cities develop and a good degree of replay-ability as a result.

But the geography is very uneven. You will be resource-constrained throughout much of the game so having the cash on hand to build through coastal mountain regions (or gather resources buried in their foothills) can be expensive the moment you move away from the narrow strip of relatively-even coastal ground.

I originally put this in "C" but I ended up moving this to "A" tier as I went through it for review. The artwork really isn't bad and I really do appreciate the novelty of the "settlement" mechanic and the way it allows a degree of customization.

Canada ("Great Lakes")

Half of this map is going to be familiar from the core game. You can tell how much of the original northeast and midwest scenarios are reused here for the United States portion of the map. But being able to build out heavy arteries along the St. Lawrence channel is rewarding and the mechanic (a snow-storm that slows down a significant portion of trains in the northern part of the map) adds an interesting twist.

The goods are also interesting and a "trading center" mechanic gives you the flexibility to get key resources even though they aren't available (or producible) domestically. But the scenario itself (even when not playing "easy" difficulty) is pretty "meh". You don't have the same sense of growth and "unlocking" you get from the Australia DLC. The beginning is a little bit tricky (though not crazy, you are just very constrained with respect to the resources you have access to) but once you get your arteries going and a portion of the resources are unlocked, you can sleep-walk your way to presidency.

I ended up putting this one in "C" tier. It wasn't challenging or innovative and I simply didn't get the same sense of buy-in.

France

Some good things about this scenario: While there are some interesting historical aspects to the scenario story line, the big winning point is the fact that you choose partway through between two branches or alliances. This determines which direction of the map you migrate towards during the middle portion of the scenario, as well as how difficult your final objectives are.

But even then (and with the "region" unlocking mechanic), you don't use much of the map, and the portion you do use doesn't force you to grow much or include much consideration for the topography, which is particularly bland. There are a few interesting places along the edges (including a criminally-underutilized northeast corner, where you practically ignore Belgium; Luxembourg; and most of Alsace-Lorrainee) but you never really "have" to play through them and what's left simply isn't challenging.

I almost dropped this into "F" tier because the game play is so blasé. I ended up putting it in "C" tier because the "split" scenario story line is redeeming. There are also interesting variations on resources. (Whereas wine is a rare / higher-tier resource in Australia, it is a much lower "fundamental" tier in France. THE FRENCH NEED THEIR WINE! At least according to the developers.)

Germany (and surrounding areas)

"Patchwork" is the name of this scenario, so it's no surprise you have to "unlock" access to different regions. But Germany has a great cross-section of historical events during the unification of the country from various regional principalities--and the railway (in conjunction with some interesting historical characters) plays a key historical role in how this region is unified.

The map is also huge, with a significant variety (while well-distributed) of topography. You also learn a lot of interesting geography--different regions and cities across Germany (and surrounding countries, like Poland and Austria).

The scenario is also challenging--satisfyingly so. It gets quite difficult in the second half, and by the end (after many unsuccessful attempts) I found myself compiling spreadsheets that tracked the growth of different cities to hit the final objective. There are 10 cities that have to grow beyond 120,000 residents, which (in this game) is a big reach that requires a lot of coordination and "logistical nuking" of specific cities with multiple warehouses and dedicated lines. Careful planning, some luck with promoters, and consideration of how key goods can move around quickly within critical "clusters" of cities ends up being very important.

I had to go pretty hardcore to beat this one, but it was very rewarding. I bumped this one straight to the top--"S" tier all the way.

Japan

First thing about Japan--for one DLC, you get three maps (all of Honshu and Hokkaido; northern Honshu; and southern Honshu) and two scenarios. The scenarios focus on earlier Japan (northern Honshu) as the emperor decides to open up and industrialize the country, and later Japan (southern Honshu) as you attempt to consolidate private networks into a national rail line.

The geography is definitely unique. Going from Tokyo down to Kyoto is not easy--this is a mountainous island chain, after all. There is also a "traveler" mechanic that looks at specific properties of particular routes and where passengers "want" to go (based on attractions and other factors).

But the thing that stands out most in this DLC is the degree of art customization, which blows every other DLC out of the water. The player avatars are all customized (Don Lorenzo is a sumo wrestler, for example) with their own Japan-related twist, and the resources are almost completely unique. The objectives are challenging, the region-unlocking patterns are unique, and the landscape artwork (including the towns, engines, and forests of seasonally-pink cherry trees) are simply wonderful. You can tell this DLC was a real tribute and act of love.

Of course there are fundamental limits (even for the more focused maps) imposed by a linear geography, so you don't get as great a use of the overall "space" within the map (e.g., compared to the more rectangular Germany). But that's the worst thing I can say about a great DLC. "S" tier, all the way. Wonderful stuff.

Sweden (Scandinavia)

This map includes middle & southern Sweden, a corner of Norway, and almost all of Denmark. There is a decent mix of topography (though building routes out to Denmark are pretty fixed, and the mountains in Norway just saturate the topography with challenges). There is also a "snow" mechanic (like Canada), but unlike Canada it covers a significant portion of the map (not just the northern "slice"). Norway regions, for example, are practically unusable during the winter. In fact, the snow mechanic is probably too much--instead of penalizing trains to 5% (or so) speed, it should probably be 20% or 25%.

But there are some redeeming aspects. There is some insightful and humorous dialog from your advisor, and interesting consideration of historical context. But the scenario itself is very "meh" and unremarkable. The final objectives are both optional and practically impossible (with very strict time limits). Expansion into Norway is inevitable and by the time you get access to Denmark (a full 30% or so of the build-able map area) the scenario is basically over.

For that reason, I put this one in "C" tier.

GB/UK (Ireland, Scotland, England, and Wales)

The fact that these scenarios give you a historical narrative is nowhere more obvious than in this scenario. The super-basic locomotives you start with are literally some of the earliest trains that ever existed. There's a real sense of history in building out the very first rail networks, and a consideration of how important those events were for the industrialization of England in particular.

The resources aren't bad, and there are a lot of custom engines. But the geography is poorly utilized and it's hard to look past how irrelevant (both difficult and not worth the mediocre reward) Ireland and Scotland are. (Did you know you could build a bridge across the Sea of Ireland!? Apparently this is a thing.) This geography is horribly utilized everywhere outside of England, and the scenario itself never reaches beyond that area.

They probably should have either done three separate maps (where you could better expand upon and utilize the unique geography in Scotland and Ireland) or purely focus on England and Wales itself. This would make better use of the more-rectangular area of central England, because you can't really do anything productive in the northern and western areas of the map.

The scenario itself is challenging-ish; it's one of the first scenarios where you really have to think about deliberate growth for a city beyond 100,000 (needing to coordinate multiple stations, etc.). This is a good DLC but it could have been a lot better. The historical context and unique engines are great though, so overall I ended up putting this one in "A" tier.

Mexico

This scenario is fun. It doesn't have the challenge or efficient utilization of size & geography that Germany does. But it's a great scenario that gives you appreciation of the complex topography of Mexico and the challenges involved in building out a Mexican rail network.

Another entertaining part of this scenario is the choice or "split" scenario that forces you to choose (largely as a function of where you deliver key resources) between two different historical allies, which has an impact on where you grow and how you place an emphasis on different resources. Both allies also come with an entertaining script of dialog.

Another interesting part of the game is the cross-border relationship with cities and other locations in Texas & the southwest US. You get a real appreciation for key cities like Monterey, even as regions like the Yucatan are relatively underutilized, across an interesting path of topography.

For these reasons I put this in as a solid "A". There isn't a lot of customization of artwork and locomotives, and not a lot of competition with other rail networks. But the "split" choice does give you a degree of replay-ability that you wouldn't otherwise get from the challenge of the scenario itself (it wasn't the easiest one, but I definitely wasn't whipping out my Germany spreadsheets!)

Winners and Losers

Of course, we haven't dropped anything into "F" tier. But I'll force myself to choose a winner from "S" tier and a loser from "C" tier.

Between Germany and Japan, I'd select Germany as the winner. The map area is just much better utilized and as a result the topography (which is already more balanced) feels much bigger.

Between the "C" tier candidates, I was torn between choosing Canada and France. I don't think either one is horrible, but in the case of Canada the scenario simply wasn't that challenging or interesting, and at least in France you had the "split" scenario story line, whereas in Canada you didn't even have that much of a unique map to play with. Sorry Canada, but into last place you go.

Conclusion

While it has a learning curve, Railway Empire is a fantastic game if you're looking for a less-linear Factorio alternative with some interesting educational twists. It's a classic spiritual successor to the Tycoon series and the DLCs are a work of a love for trains that really shines through. You really do get something for your money.

[22/52] Random Software Project: dev.to Frontend Conclusion

Brian Kirkpatrick — Wed, 09 Oct 2024 18:13:27 +0000

It's Over, Man!

We've reached the final episode of our three-parter on what was once the dev.to frontend challenge. We've gotten quite distracted, haven't we!? While it's far too late to submit for the challenge itself, we've learned some good lessons along the way--and even published a neat little library. But now it's time to wrap it up.

We'll focus mainly on two big things today: integrating our solarplanets library to compute the actual 3d positions of our planets in realtime, and finishing the integration of the markup from the original challenge via on-click events for the individual planet meshes.

Integrating Solar Planets

You might recall from part two that we had put together a decent little solarplanets library, based on a useful algorithm from one of my go-to astro texts. We've since used a basic yarn publish to put this up on npmjs.org for easy access; it even has its own project page now! Cool.

We can start with a basic yarn add solarplanets, then, from where we left off in part one. Then, we can call yarn run dev to get a live version of our app going. And now we're ready to use it!

Compute Planet Positions

After importing the library, we'll also need the data (or "catalog") of planetary orbit tables to pass to the function call for each planet. We called this the "standish catalog" after the original paper where it was published. We'll copy it over to load as a resource in our app (just a raw import for now is fine), then parse it on application load.

Now, we can update our "planet factory" (buildPlanet()) to include a "name" parameter (so we can correlate against this catalog). Then, we'll use the getRvFromElementsDatetime() method from our library, if a match is found, to determine the position we assign to that mesh.

const poe = window.app.catalog(hasOwnProperty(name.toLowerCase())
    ? window.app.catalog[name.toLowerCase()]
    : null;
if (pow) {
    const [r, _] = solarplanets.getRvFromElementsDatetime(poe, window.app.now);
    mesh.position.set(
        spatialscale * r[0],
        spatialscale * r[1],
        spatialscale * r[2]
    );
}

We'll also add this window.app.now Date object, though there is also a newer THREE.Clock feature we could use for more cohesive time modeling. And once we've assigned those positions (updating the call to this factory to include the new name parameter as well, of course, in addition to parsing the catalog JSON on window load), we can now see our planetary positions!

Sampling Orbit Traces

One thing you may notice is, while our planets have "real" positions now, they don't match the "orbit traces" we had produced lines for. Those are still implemented as "flat" circles, so it's no surprise they don't line up. Let's fix that now.

To compute the orbit trace for a planet, we'll use a similar function call but we'll sample over time from "now" until one orbital period in the future. This orbital period, though, is different for each planet, so we'll need to look at our catalog again. Let's add a "name" parameter to the orbit trace "factory" as well, then dive in and update how those points are being computed.

First, we'll replace the "uniform" sampling with a sampling across time. We'll still use a fixed number of points, but we'll interpolate those from the "now" time to one orbital period into the future. We'll add a helper method to compute this "orbital period" from the catalog model entry, which is a pretty straightforward equation that we've actually looked at in previous articles.

function getPeriodFromPoe(poe) {
    const MU_SUN_K3PS2 = 1.327e11;
    const au2km = 1.49557871e8;
    return 2 * Math.PI * Math.sqrt(Math.pow(poe.a_au * au2km, 3.0) / MU_SUN_K3PS2);
}

Now we'll use this to change the for loop in the buildOrbitTrace() function.

const poe = window.app.catalog.hasOwnProperty(name.toLowerCase())
    ? window.app.catalog[name.toLowerCase()]
    : null;
if (pow) {
    const T0 = window.app.now;
    const T_s = getPeriodFromPoe(poe);
    for (var i = 0; i < (n + 1); i += 1) {
        const t_dt = new Date(T0.valueOf() + T_s * 1e3 * i / n);
        const [r, _] = solarplanets.getRvFromElementsDatetime(poe, t_dt);
        points.push(new THREE.Vector3(
            spatialScale * r[0],
            spatialScale * r[1],
            spatialScale * r[2]
        ));
    }
}

And with that you should see some 3d orbit traces now! They're slightly tilted, they're slightly skewed, and they line up with our planet positions. Pretty darn cool.

Finishing Raycaster Collision Detection

When we left off in part one, we had just put in a raycaster to evaluate (on each frame) collisions with meshes in our scene. I have one brief update, which is that the mouse position was not being translated correctly. There was a sign error in the calculation for the y component, which should instead look something like this:

window.app.mouse_pos.y = -(event.clientY / window.innerHeight) * 2 + 1;

With that fixed, let's debug / verify that these collisions are actually taking place. When intersections are computed (in our animate() function), we're going to pass those intersections off to a handler, processCollision(). We're also going to count how many intersections are triggered, so we know when to "reset" the mouseover effects we're going to add.

let nTriggered = 0;
if (intersections.length > 0) {
    intersections.forEach(intersection => {
        nTriggered += processCollision(intersection) ? 1 : 0;
    });
}
if (nTriggered === 0) {
    // reset stuff
}

With that in mind, we're ready to write our handler. First, let's just check to make sure the mesh has a name. (Which will require, by the way, that we assign the planet name in the buildPlanet() factory to the mesh that is created. In this function, we will also need to set a scale for the mesh, like mesh.scale.set(2, 2, 2), to make collisions easier. Make sure you make these changes now.) If it has a name that matches a known planet, we'll log it to the console for now.

function processCollision(intersection) {
    const name = intersection.object.name;
    const names = planets.map(p => p.name.toLowerCase());
    if (name !== "" && names.includes(name)) {
        console.log(name);
        return true;
    }
    return false;
}

(Note we are normalizing names to lower case for comparison--always a good idea, just in case you lose track of consistent capitalization within the model data you are using.)

This should be enough for you to see (for example) "venus" should up in your console log when you move the mouse over Venus.

Trigger Mouseover Effects

With intersections being computed correctly, let's add some effects. Even if the user doesn't click, we want to indicate (as a helpful hint) that the mouse is over something clickable--even though it's a mesh in a 3d scene, and not just a big button in a 2d UI. We're going to make two changes: the mesh will turn white, and the cursor will change to a "pointer" (the hand, instead of the default arrow).

In our processCollision() function, we'll replace the console.log() call with a change to the object material color, and change the body style of the document (a little cheating here...) to the appropriate style.

//mouseover effects: change planet color, mouse cursor
intersection.object.material.color.set(0xffffff);
window.document.body.style.pointer = "cursor";
return true;

We'll also need to use our collision check aggregation (nTriggered, in the animate() function) to reset these changes when the mouse is not hovering over a planet. We had an if block for this, let's populate it now:

if (nTriggered === 0) {
    // reset state
    window.document.body.style.cursor = "inherit";
    planets.forEach(p => {
        const sgn = window.app.scene.getObjectByName(p.name.toLowerCase());
        if (sgn) {
            sgn.material.color.set(p.approximateColor_hex);
        }
    });
}

Give it a spin! Hover over some planets and check out that deliciousness.

Trigger On-Click Dialog

To satisfy the original intention (and requirements) of the challenge, we need to integrate the markup. We'll move all of this markup into our HTML, but tweak the style to hide it so the 3d canvas continues to take up our screen. Instead, when a planet is clicked, we'll query this markup to display the correct content in a quasi-dialog.

If you copy the entire body of the markup, you can tweak our top-level CSS to make sure it doesn't show up before a mouse button is pressed:

header, section, article, footer {
    display: none
}

And from the console, you can verify that we can use query selector to grab a specific part of this markup:

window.document.body.querySelector("article.planet.neptune");

So, we're ready to implement our on-click event. But, we need to keep track of the mouse state--specifically, if the button is down. We'll add an is_mouse_down property to our module-scope window.app Object, and add some global handlers to our onWindowLoad() function that update it:

window.addEventListener("mousedown", onMouseDown);
window.addEventListener("mouseup", onMouseUp);

The sole purpose of these event handlers is to update that window.app.is_mouse_down property so we'll be able to check it in our collision intersection handler. So, they're pretty simple:

function onMouseDown(event) {
    window.app.is_mouse_down = true;
}

function onMouseUp(event) {
    window.app.is_mouse_down = false;
}

In a more formally designed application, we might encapsulate management (and lookup) of mouse state (and signals) in their own object. But this will work for our purposes. Now, we're ready to revisit the body of our processCollision() handler and add support for showing the dialog:

if (window.app.is_mouse_down) {
    dialogOpen(name);
}

We'll add two handlers for dialog management: dialogOpen() and dialogClose(). The latter will be invoked when the mouse moves out of the mesh collision--in other words, exactly where we reset the other UI states, in animate():

if (nTriggered === 0) {
    // ...
    dialogClose();
}

Now we're ready to write these. Since we're not using any kind of UI library or framework, we'll manually hand-jam a basic <div/> that we populate from the article content, then style as needed.

function dialogOpen(name) {
    let dialog = window.document.body.querySelector(".Dialog");
    if (!dialog) {
        dialog = window.document.createElement("div");
        dialog.classList.add("Dialog");
        window.document.body.appendChild(dialog);
    }

    // query article content to populate dialog
    const article = window.document.body.querySelector(`article.planet.${name}`);
    if (!article) { return; }
    dialog.innerHTML = article.innerHTML;
    dialog.style.display = "block";
}

A word of caution: Assigning innerHTML is a surprisingly expensive operation, even if you're just emptying it with a blank string! There's all sorts of parsing and deserialization the browser has to do under the hood, even if it is a method native to the DOM API itself. Be careful.

Conversly, the dialogClose() event is quite simple, since we just need to check to see if the dialog exists, and if it does, close it. We don't even need to know the name of the relevant planet.

function dialogClose() {
    const dialog = window.document.body.querySelector(".Dialog");
    if (!dialog) { return; }
    dialog.style.display = "none";
}

And of course we'll want to add some modest styling for this dialog in our top-level CSS file for the application:

.Dialog {
    position: absolute;
    top: 0;
    left: 0;
    background-color: rgba(50, 50, 50, 0.5);
    color: #eee;
    margin: 1rem;
    padding: 1rem;
    font-family: sans-serif;
    border: 1px solid #777;
    border-radius: 1rem;
}

Wrapping It Up

Of course, we still only have the "inner" planets. We can easily add the "outer" planets just by augmenting our module-scope planets dictionary that is used to populate the scene. (Entries in the markup and the planet catalog already exist.)

// ... }, 
{
    "name": "Jupiter",
    "radius_km": 6.9911e4,
    "semiMajorAxis_km": 7.78479e8,
    "orbitalPeriod_days": 4332.59,
    "approximateColor_hex": 0xaa7722
}, {
    "name": "Saturn",
    "radius_km": 5.8232e4,
    "semiMajorAxis_km": 1.43353e9,
    "orbitalPeriod_days": 10755.7,
    "approximateColor_hex": 0xccaa55
}, {
    "name": "Uranus",
    "radius_km": 2.5362e4,
    "semiMajorAxis_km": 2.870972e9,
    "orbitalPeriod_days": 30688.5,
    "approximateColor_hex": 0x7777ff
}, {
    "name": "Neptune",
    "radius_km": 2.4622e4,
    "semiMajorAxis_km": 4.50e9,
    "orbitalPeriod_days": 60195,
    "approximateColor_hex": 0x4422aa
}

This is even enough for you to start doing some verification & validation when you walk outside tonight! Take a close look at how the planets are arranged. At midnight, you're on the far side of the earth from the sun. What planets should you be able to see, if you trace the path of the zodiac (or ecliptic) from east to west? Can you see Venus, and when? What's the relative arrangement of Jupiter and Saturn? You can predict it!

Cool stuff.

[21/52] Random Software Project: Solar Planets

Brian Kirkpatrick — Sat, 28 Sep 2024 05:08:32 +0000

What was once a one-parter intended to turn around over the weekend for a challenge submission has turned into a three-parter with a brief interlude for some numerics! What could be tastier?

What Are We Doing Here?

This is effectively part two of our analysis regarding how the frontend challenge from the other week on dev.to could be realized using THREE.js and a little bit (okay, maybe too much) of knowledge from the aerospace world.

Specifically, if you recall our previous article: we've reached a place where we have 3d planets around a sun--but they're all in a flat circle, perfectly lined up with each other (even if they're realistically ranged apart). We want them to be moving through 3d space, and it would be even better if we could compute those 3d positions from the actual orbits of the planets themselves in realtime!

This is entirely possible thanks to a neat high-power trick I like to call: "implementing algorithms and test cases from a handy textbook". (This is a great exercise and an effective way to build up a library of really useful, well-proven tools, while simultaneously sharpening your own knowledge.) In particular, we'll be using one of my go-to textbooks, Curtis's "Orbital Mechanics for Engineering Students". He has a chapter on planetary trajectories, including a subsection specifically on computing planetary ephemerides from a known table or model of orbital elements and their drift.

Fun side note: you can read the original paper on which his approach is based, including the derivation of the equations and how his numerical data was put together. Good stuff! If you wanted higher-fidelity models, you would do something like evaluting the 17th-order polynomials from a JPL Spice kernel. But this approach, where we use a mean-drift rate of planetary orbital elements about an established epoch, is still pretty good.

But, as a side bonus, this also lets us explore a useful pattern for single-file Javascript modules. We've covered this briefly in some previous articles, before we started doing engineering content, but this will let us demonstrate how a real package can be published and automated. We'll include documentation, testing, coverage, and a degree of type-hinting, in addition to demonstrating the software engineering process itself.

Let's Get It Started

So where do we begin? Let's start with an empty yarn project and git-init it:

yarn init -y .
git init .
git add -A
git commit -m "initial content commit"

Then, we'll crack open a new index.mjs and get going.

But get going doing what?

Recall, our intended purpose is to compute the position of the planets at any given point in time. Effectively, we want a vector as a function of the planetary element model and a specific Date value. We've already seen what that table might look like, but we'll want this data to be usable in a self-contained Javascript module. Here's a part of the table from the original paper:

For the purposes of avoiding dependencies, this means we'll transcribe it into a standish_catalog.json file. From the Curtis text, that means different entries for each planet that look something like this:

{
    "mercury": {
        "a_au": 0.38709893,
        "da_au": 0.00000066,
        "e": 0.20563069,
        "de": 0.00002527,
        "inc_deg": 7.00487,
        "dinc_sec": -23.51,
        "raan_deg": 48.33167,
        "draan_sec": -446.30,
        "lop_deg": 77.4545,
        "dlop_sec": 573.57,
        "ml_deg": 252.25084,
        "dml_sec": 538101628.29
    },
    ...
}

What's going on here?

The Model

We have six values, a variation on the classical orbital elements, for each planet. Those values are evaluated at a specific epoch, J2000 (noon UTC on January 1st, 2000). We also have a "rate" for each value, the mean rate of change per Julian century. (These elements don't change very quickly! But they do change, or "drift".)

a_au is the "semi-major axis". Every orbit makes an ellipse, and this effectively measures the scale of the ellipse. In particular, this value has units of au, or "astronomical units". 1 [au] is roughly equal to the distance between the earth and the sun.
e is the "eccentricity", which measures how elliptical (or lop-sided) the orbit is. If this is 0, the orbit is a circle. If this is 1, the orbit is a parabola. But, this value for planets will typically be somewhere between 0 and 1--an ellipse.
inc_deg is the "inclination" of the orbital plane with respect to some reference--in our case, the inertial heliocentric ecliptic coordinate frame. Sometimes you will hear this referred to as "ICRF" or "ICRS".
raan_deg defines the "right-ascension of the ascending node". This is the angle from some reference (the X vector) rotated about the Z vector until we meet the point at which the orbital plane intersects (in ascension, as opposed to descension) our X-Y reference plane.
lop_deg is "longitude of perhelion", which is the sum of two measurements: the ascending node (see above) and the "argument of perhelion", which is the angle (within the orbital plane) from the ascending node to perhelion, the point of closest approach to the sun.

ml_deg is the "mean longitude" is a similar sum of angles, but instead for the "mean" (constant rate in time) longitude of the body in question at some point in time (in other words, our epoch).

For a specific point in time, we will compute the values for each of these elements as a linear "drift" from our epoch data. For a_au, this means a drift rate of astronomical units per Julian century. For e, this means a drift rate of eccentricity (which is a unitless ratio) per Julian century. For all other values, which are angles, we express this drift rate in "seconds" (as in, degrees/minutes/seconds) per Julian century.

The Algorithm

We have enough information now to construct the "body" or top level of our algorithm. There will be plenty of opportunities to fill in the gap with various utility functions, but I find it helps to first define the "entry points", or main functions an external user might call, and then decompose into smaller behaviors before you begin implementation from the bottom-up.

So what does the top-level call look like? Let's use this to start writing our index.mjs content.

/**
 * Computes the position and velocity of a given planetary model as evaluated at the given datetime. Results are returned in a "sun-centered inertial", which is defined in the heliocentric ecliptic frame of reference.
 * 
 * @param {Object} planetaryOrbitalElements - Planetary orbital elements model, including mean (offset and rate) coefficients
 * @param {Date} dt - Datetime at which state will be evaluated
 * @returns {Array} - Two-element array containing r and v as evaluated in the heliocentric ecliptic frame; each element is a three-element array of numeric values in [km] and [km/s], respectively
 */
export function getRvFromElementsDatetime(planetaryOrbitalElements, dt) {
}

A couple of notes here:

We'll be exporting everything, mainly because to want to thoroughly test each "level" of our algorithm. More on that later.
You'll note some nice JSDoc-style header comments here. VS Code in particular does a good job of auto-expanding these above a function header once you start with a /**. You can even use JSDoc parameter and return annotations to perform a degree of type-checking in vanilla Javascript. But I mainly like to use it to remind myself of exactly what arguments and return values are supposed to be.
We're using our own internal / native Javascript Arrays (and Arrays-of-Arrays) for linear algebra types like vectors and matrices, respectively. In this case, our return position and velocity are each three-element Arrays. You could also use a numerical library (like Sylvester) or reuse the linear algebra types from THREE.js, but for the purpose of being self-contained and dependency-free (for runtime at least), this is good for now.

We'll break down the body of this function around the steps described in Curtis's algorithm ("Algorithm 8.1", if I remember correctly).

The Implementation

First, we need to compute the julian date value from the julian date number and universal time. We'll also convert this into julian centuries from J200 so we can compute our orbital elements.

// determine julian date value and julian centuries
const y = dt.getUTCFullYear();
const m = dt.getUTCMonth() + 1; // thanks, javascript!
const d = dt.getUTCDate(); // THANKS, JAVASCRIPT!!!
const H = dt.getUTCHours();
const M = dt.getUTCMinutes();
const S = dt.getUTCSeconds() + dt.getUTCMilliseconds() * 1e-3;
const J0 = getJ0FromDatevec(y, m, d);
const UT = getUtFromTimevec(H, M, S);
const JD = getJdFromDateTime(J0, UT);
const T0 = getJulianCenturies(JD);

Note that we've defined some utility functions here that we'll implement later, because we know roughly what the call signature needs to be. Next, we need to interpolate the elements themselves. It would help here to use some generic linear interpolation function--in this case, we'll call it getCurrentElements(). We'll also define some unit conversion constants. By the time we've computed the elements, we want them to be in SI units for internal representation (note the unit abbreviations following the underscores in variable names), and for angles that includes a positive-modulus function for wrapping within the range [0, 2 * Math.PI].

// interpolate (and convert) planetary elements
const d2r = Math.PI / 180;
const au2km = 1.49597871e8;
const a_km = getCurrentElements(planetaryOrbitalElements.a_au * au2km, planetaryOrbitalElements.da_au * au2km, T0);
const e = getCurrentElements(planetaryOrbitalElements.e, planetaryOrbitalElements.de, T0);
const inc_rad = posmod(getCurrentElements(planetaryOrbitalElements.inc_deg * d2r, planetaryOrbitalElements.dinc_sec / 3600 * d2r, T0), 2 * Math.PI);
const raan_rad = posmod(getCurrentElements(planetaryOrbitalElements.raan_deg * d2r, planetaryOrbitalElements.draan_sec / 3600 * d2r, T0), 2 * Math.PI);
const lop_rad = posmod(getCurrentElements(planetaryOrbitalElements.lop_deg * d2r, planetaryOrbitalElements.dlop_sec / 3600 * d2r, T0), 2 * Math.PI);
const ml_rad = posmod(getCurrentElements(planetaryOrbitalElements.ml_deg * d2r, planetaryOrbitalElements.dml_sec / 3600 * d2r, T0), 2 * Math.PI);

Next, we'll compute angular momentum (the scalar, but still an incredibly helpful value), as well as the argument of perhelion and mean anomaly.

// compute angular momentum, argument of periapsis, and mean anomaly from elements
const h_km2ps = getAngularMomentum(a_km, e);
const aop_rad = getAopFromLopRaan(lop_rad, raan_rad);
const M_rad = getMaFromMlLop(ml_rad, lop_rad);

Again, we have some utility functions we know we'll want later. They aren't complicated, but pulling them apart (even if we don't reuse them) will give us useful "units" of code against which we can write unit tests. We're now ready for the "tricky" part--solving Kepler's equation, a transcendental equation that must be solved numerically (in this case using the Newton-Raphson method).

// solve kepler's equation for true anomaly
const E_rad = getEaFromMaE(M_rad, e);
const tht_rad = getTaFromEaE(E_rad, e);

Once we have "true anomaly", we've finished collecting all the elements we need. We can now compute the planet's coordinates, first in a "perifocal" plane. The perifocal plane is fixed with respect to the ellipse, with the ellipse in the X-Y plane and -X pointing towards periapsis (+X points towards apopasis). These position and velocity vectors are easily determined from angular momentum, eccentricity, and the true anomaly.

// compute perifocal states
const rPqw_km = getRpqwFromHETht(h_km2ps, e, tht_rad);
const vPqw_kmps = getVpqwFromHETht(h_km2ps, e, tht_rad);

Now that we have vectors in the perifocal plane, we simply need to transform it back into the heliocentric equinoctal frame. This is done with a "3-1-3" series of rotations, using the other three angles we computed, concatenated into a frame transformation we'll call Qpqw2eci (though we're not in ECI, which is earth-centered, the rotation transformations are the same). Once computed, we can multiply our vectors by this frame transformation to realize the vectors in an "inertial" coordinate frame.

// convert to inertial states and return
const Qpqw2eci = getQpqw2eci(raan_rad, inc_rad, aop_rad);
const rHcec_km = getMatVec(Qpqw2eci, rPqw_km);
const vHcec_kmps = getMatVec(Qpqw2eci, vPqw_kmps);

(Note in this case we use getMatVec(), a utility function that evaluates a matrix multiplied by a vector. There are a variety of linear algebra operations we need to implement ourselves, since we're using native arrays and not a third-party math library, but they're not particularly complicated.)

return [
    rHcec_km,
    vHcec_kmps
];

Finally, we return this pair of vectors from the function--and we're done!

A Word About Utility Functions

We won't walk through every utility function here--you can always see the source index.mjs in the GitHub project. But there are a few worth a closer look.

Here's our posmod() function, which we use because the native Javascript modulus operator (say, for x % X) does not return a value between 0 and X, but rather between -X and +X. We want to "clamp" several values (mostly angles) to a positive range instead.

/**
 * Floating-point modulus that evaluates a % b to the positive range [0,b)
 * 
 * @param {Number} a - Numerical subject value
 * @param {Number} b - Modulo
 * @returns {Number} - a mod b within [0,b)
 */
export function posmod(a, b) {
    const c = a % b;
    return c < 0 ? c + b : c;
}

Here's probably the "meatiest" function in the bunch, which uses the Newton-Raphson method to find the root of our transcendental Kepler's function. This helps us solve for "eccentric" anomaly from mean anomaly and eccentricity, after which we can easily compute "true" anomaly.

/**
 * Computes eccentric anomaly from mean anomaly and eccentricity. This implements equation Eq. 3.14, solving Kepler's equation numerically via Alg. 3.1.
 * 
 * @param {Number} M_rad - Mean anomly [rad]
 * @param {Number} e - Eccentricity
 * @returns {Number} - Eccentric anomaly [rad]
 */
export function getEaFromMaE(M_rad, e) {
    let E_rad = M_rad < Math.PI ? M_rad + 0.5 * e : M_rad - 0.5 * e;
    let isConverged = false;
    let isOverrun = false;
    let n = 0;
    const nMax = 1e3;
    const tol = 1e-8;
    while (!isConverged && !isOverrun) {
        const f = E_rad - e * Math.sin(E_rad) - M_rad;
        const df = 1 - e * Math.cos(E_rad);
        const r = f / df;
        n += 1;
        isConverged = Math.abs(r) < tol;
        isOverrun = nMax < n;
        if (!isConverged) {
            E_rad = E_rad - r;
        }
    }
    return E_rad;
}

Of course, there's no guarantee we will converge! But for planets, it is highly likely, since the elements we are dealing with are fairly mild (compared to edge cases). Lastly, it's also worth taking a look at how we compute perifocal position, which is surprisingly simple but illustrates how we can handle Arrays as linear algebra types with a little help from judicious JSDoc comments:

/**
 * Computes position in perifocal (PQW) frame given angular momentum, eccentricity, and true anomaly.
 * 
 * @param {Number} h_km2ps - Angular momentum scalar [km^2/s]
 * @param {Number} e - Eccentricity
 * @param {Number} tht_rad - True anomaly [rad]
 * @returns {Array} - Position in PQR frame [km]
 */
export function getRpqwFromHETht(h_km2ps, e, tht_rad, mu_km3ps2 = MU_SUN_KM3PS2) {
    const c = h_km2ps * h_km2ps / (mu_km3ps2 * (1 + e * Math.cos(tht_rad)));
    return [
        c * Math.cos(tht_rad),
        c * Math.sin(tht_rad),
        0
    ];
}

Does It Work?

One of the great things about implementing an algorithm from a textbook is that you have all sorts of great reference cases to build a test suite from. In this case, this is true for the top-level algorithm and practically every function underneath it. Let's add a test.mjs file and look at how this could be done.

We'll use Jasmine, a useful test framework for Javascript that works in a variety of contexts, has an elegant declaration style, and includes support for several widely-compatible report formats. From our test module, we'll export a key-value pair of test descriptions and test methods. Here's a super-simple "smoke test" that simply ensures that the single-file module can be imported and that tests can be run.

import * as solarplanets from "./index.mjs";

export default {
    "can be tested": () => {
        expect(true).toEqual(true);
    }
}

Of course, we'll want to add more than this! We want to test every line of code (or at least every function call). This will include a combination of "examples" we reproduce from the textbook (which we'll reference in the test description) and specific tests for common foot-guns like calculation of rotation matrices (vs. frame transformations). For example, here's "Exercise 5.6" from the Curtis text, reproduced as another test entry in our tests.mjs module.

...
    "can reproduce example 5.6 T0": () => {
        const J0 = solarplanets.getJ0FromDatevec(2004, 3, 3);
        const UT = solarplanets.getUtFromTimevec(4, 30, 0);
        const JD = solarplanets.getJdFromDateTime(J0, UT);
        expect(solarplanets.getJulianCenturies(JD)).toBeCloseTo(0.041683778, 1e-3);
    },
...

And here's how we verify we didn't get our rotation matrices mixed up!

    "can rotate vector about x": () => {
        const actual = solarplanets.getMatVec(solarplanets.r1(0.5 * Math.PI), [0, 1, 0]);
        const expected = [0, 0, 1];
        for (let i = 0; i < 3; i += 1) {
            expect(actual[i]).toBeCloseTo(expected[i], 1e-3);
        }
    },

All in all, we have about 325 lines of tests, compared to about 417 lines of source in our index.mjs. I find this is about right--you want good coverage but if you start approaching the point where the SLOC for tests is greater than the SLOC for the source it evaluates, then you might want to take a step back and think about how prudently you are evaluating your code. It's easy to go overboard--think about what you really need to assert in your algorithm, versus what you're just writing for the purpose of writing predetermined tests (or for squeezing that last percentage of test coverage--we've all been there!)

Package Scripts

Let's revisit our package specification. First we need to add some development dependencies (the only ones we have, since the module implementation is self-contained). This includes development tools for testing, documentation, and coverage. We'll also dictate specific versions here because my favorite JSDoc template is a little bit picky about compatibility.

yarn add -D c8@7.12 foodoc@0.0.9 jasmine@4.4 jasmine-reporters@2.5 jsdoc@3.6 semver@7.3.8 terser@5.15

And now for the magic. We'll use some neat Javascript package tricks I've written about before to include all of our test, documentation, and coverage calls and configuration internally to our package.json. For each script (docs, test, and cov), we'll do something similar:

Extract key values from package specification, including custom blocks of properties, to write out to gitignore'd dot files like .jsdoc-conf.json
Map in shared values where necessary, like package name, before those contents are written out to ensure they are only defined in one place
Invoke those specific functions against those files and other relevant arguments

In addition to the scripts block, that also means we'll be adding a .jsdoc-conf block, a .jasmine-tests block, a .jasmine-conf block, and a .c8-conf block. (In the case of .jasmine-tests, we're actually writing out the lines of a top-level test module that iterates through our test exports; all the others are just writes of equivalent JSON). Feel free to look at the article linked above, or the source for this project, for an example.

Once that's done, we can run each one and look at the output:

yarn run test will use the dynamically-written runner to invoke each of our test cases and report the results to STDOUT
yarn run docs will generate nicely-formatted JSDoc content in the out/ folder
yarn run cov will use C8 to trace evaluation of our test suite against the source in our single-file Javascript module and determine how well our tests "cover" our source

Publishing and Conclusion

Believe it or not, we're ready to publish the package. We may need to set up our npmjs.com credentials in an RC file, but once we do, and have finished filling out the "metadata" in your package.json (like name, version, dependency, etc.), a simple yarn publish command will do the trick. And we're done! Look at that.

Of course, there's a lot more you can do with that, including CI support for automating various stages of our build. But we're at a point now where we're ready to just yarn add this new dependency to our solar system application, at which point we have a robust way to compute the realtime position of each planet. Cool beans!

[20/52] Random Software Projects: dev.to Frontend Challenge

Brian Kirkpatrick — Thu, 12 Sep 2024 03:12:46 +0000

We're going to use the current dev.to frontend challenge as a means to explore how to quickly put together a basic static file web app for 3d visualization. We'll use THREE.js (one of my favorite libraries) to put together a basic solar system tool that can be used to display the markup input from the challenge.

The Vision

Here's the current dev.to challenge that inspired this project:

https://dev.to/challenges/frontend-2024-09-04

So, let's see how quickly we can put together something along these lines!

Getting Started

From a brand-new Github project, we'll use Vite to get the project up and running with hot module replacement (or HMR) out of the box for very quick iteration:

git clone [url]
cd [folder]
yarn create vite --template vanilla .

This will create a no-frameworks Vite project that works out of the box. We just need to install the dependencies, add THREE, and run the "live" development project:

yarn install
yarn add three
yarn run dev

This will give us a "live" version that we can develop against and debug in near-real-time. And now we're ready to go in and start tearing stuff out!

Engine Structure

If you've never used THREE, there's some things worth knowing.

In engine design, there are typically three activities or loops going on at any given time. If all three are done in serial, that means your core "game loop" has a sequence of three kinds of activities:

There is some sort of user input polling or events that must be handled
There are the rendering calls themselves
There is some sort of internal logic / update behavior

Things like networking (e.g., an update packet comes in) can be treated as input here, since (like user actions) they trigger events that must propagate into some update of the application state.

And of course, underlying it all there is some representation of the state itself. If you're using ECS, maybe this is a set of component tables. In our case, this starts primarily as an instantiation of THREE objects (like a Scene instance).

With that in mind, let's start writing out basic placeholders for our app.

Stripping Stuff Out

We'll start by refactoring the top-level index.html:

We don't need the static file references
We don't need the Javascript hooks
We will want a global-scope stylesheet
We'll want to hook an ES6 module as our top-level entry point from the HTML

That leaves our top-level index.html file looking something like this:

<!doctype html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Vite App</title>
  <link rel="stylesheet" href="index.css" type="text/css" />
  <script type="module" src="index.mjs"></script>
</head>

<body>
</body>

</html>

Our global-scope stylesheet will simply specify the body should take up the whole screen--no padding, margin, or overflow.

body {
    width: 100vw;
    height: 100vh;
    overflow: hidden;
    margin: 0;
    padding: 0;
}

Now we're ready to add our ES6 module, with some basic placeholder content to make sure our app is working while we clean up the rest:

/**
 * index.mjs
 */

function onWindowLoad(event) {
    console.log("Window loaded", event);
}

window.addEventListener("load", onWindowLoad);

Now we can start pulling out things! We'll delete the following:

main.js
javascript.svg
counter.js
public/
style.css

Of course, if you look at the "live" view in your browser, it will be blank. But that's okay! Now we're ready to go 3d.

THREE Hello World

We'll start by illustrating the classic THREE "hello world" spinning cube. The rest of our logic will be within the ES6 module we created in the previous stage. First we'll need to import THREE:

import * as THREE from "three";

But now what?

THREE has a specific graphics pipeline that is both simple and powerful. There are several elements to consider:

A scene
A camera
A renderer, which has (if not provided) its own rendering target and a render() method that takes the scene and camera as parameters

The scene is just a top-level scene graph node. Those nodes are a combination of three interesting properties:

A transform (from the parent node) and an array of children
A geometry, which defines our vertex buffer contents and structure (and an index buffer--basically, the numerical data defining the mesh)
A material, that defines how the GPU will process and render the geometry data

So, we need to define each of these things to get started. We'll start with our camera, which benefits from knowing our window dimensions:

const width = window.innerWidth;
const height = window.innerHeight;
const camera = new THREE.PerspectiveCamera(70, width/height, 0.01, 10);
camera.position.z = 1;

Now we can define the scene, to which we'll add a basic cube with a "box" geometry and a "mesh normal" material:

const scene = new THREE.Scene();
const geometry = new THREE.BoxGeometry(0.2, 0.2, 0.2);
const material = new THREE.MeshNormalMaterial();
const mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);

Lastly, we'll instantiate the renderer. (Note that, since we don't provide a rendering target, it will create its own canvas, which we will then need to attach to our document body.) We're using a WebGL renderer here; there are some interesting developments in the THREE world towards supporting a WebGPU renderer, too, which are worth checking out.

const renderer = new THREE.WebGLRenderer({
    "antialias": true
});
renderer.setSize(width, height);
renderer.setAnimationLoop(animate);
document.body.appendChild(renderer.domElement);

We have one more step to add. We pointed the renderer to an animation loop function, which will be responsible for invoking the render function. We'll also use this opportunity to update the state of our scene.

function animate(time) {
    mesh.rotation.x = time / 2000;
    mesh.rotation.y = time / 1000;
    renderer.render(scene, camera);
}

But this won't quite work yet. The singleton context for a web application is the window; we need to define and attach our application state to this context so various methods (like our animate() function) can access the relevant references. (You could embed the functions in our onWindowLoad(), but this doesn't scale very well when you need to start organizing complex logic across multiple modules and other scopes!)

So, we'll add a window-scoped app object that combines the state of our application into a specific object.

window.app = {
    "renderer": null,
    "scene": null,
    "camera": null
};

Now we can update the animate() and onWindowLoad() functions to reference these properties instead. And once you've done that you will see a Vite-driven spinning cube!

Lastly, let's add some camera controls now. There is an "orbit controls" tool built into the THREE release (but not the default export). This is instantiated with the camera and DOM element, and updated each loop. This will give us some basic pan/rotate/zoom ability in our app; we'll add this to our global context (window.app).

import { OrbitControls } from "three/addons/controls/OrbitControls.js";
// ...in animate():
window.app.controls.update();
// ...in onWindowLoad():
window.app.controls = new OrbitControls(window.app.camera, window.app.renderer.domElement);

We'll also add an "axes helper" to visualize coordinate frame verification and debugging inspections.

// ...in onWindowLoad():
app.scene.add(new THREE.AxesHelper(3));

Not bad. We're ready to move on.

Turning This Into a Solar System

Let's pull up what the solar system should look like. In particular, we need to worry about things like coordinates. The farthest object out will be Pluto (or the Kuiper Belt--but we'll use Pluto as a reference). This is 7.3 BILLION kilometers out--which brings up an interesting problem. Surely we can't use near/far coordinates that big in our camera properties!

These are just floating point values, though. The GPU doesn't care if the exponent is 1 or 100. What matters is, that there is sufficient precision between the near and far values to represent and deconflict pixels in the depth buffer when multiple objects overlap. So, we can move the "far" value out to 8e9 (we'll use kilometers for units here) so long as we also bump up the "near" value, which we'll increase to 8e3. This will give our depth buffer plenty of precision to deconflict large-scale objects like planets and moons.

Next we're going to replace our box geometry and mesh normal material with a sphere geometry and a mesh basic material. We'll use a radius of 7e5 (or 700,000 kilometers) for this sphere. We'll also back out our initial camera position to keep up with the new scale of our scene.

// in onWindowLoad():
app.camera.position.x = 1e7;
// ...
const geometry = new THREE.SPhereGEometry(7e5, 32, 32);
const material = new THERE.MeshBasicMaterial({"color": 0xff7700});

You should now see something that looks like the sun floating in the middle of our solar system!

Planets

Let's add another sphere to represent our first planet, Mercury. We'll do it by hand for now, but it will become quickly obvious how we want to reusably-implement some sort of shared planet model once we've done it once or twice.

We'll start by doing something similar as we did with the sun--defining a spherical geometry and a single-color material. Then, we'll set some position (based on the orbital radius, or semi-major axis, of Mercury's orbit). Finally, we'll add the planet to the scene. We'll also want (though we don't use it yet) to consider what the angular velocity of that planet's orbit is, once we start animating it. We'll consolidate these behaviors, given this interface, within a factory function that returns a new THREE.Mesh instance.

function buildPlanet(radius, initialPosition, angularVelocity, color) {
    const geometry = new THREE.SphereGeometry(radius, 32, 32);
    const material = new THREE.MeshBasicMaterial({"color": color});
    const mesh = new THREE.Mesh(geometry, material);
    mesh.position.set(initialPosition.x, initialPosition.y, initialPosition.z);
    return mesh;
}

Back in onWindowLoad(), we'll add the planet by calling this function and adding the result to our scene. We'll pass the parameters for Mercury, using a dullish grey for the color. To resolve the angular velocity, which will need to be in radius per second, we'll pass the orbital period (which Wikipedia provides in planet data cards) through a unit conversion:

The resulting call looks something like this:

// ...in onWindowLoad():
window.app.scene.add(buildPlanet(2.4e3, new THREE.Vector3(57.91e6, 0, 0), 2 * Math.PI / 86400 / 87.9691, 0x333333));

(We can also remove the sun rotation calls from the update function at this point.)

If you look at the scene at this point, the sun will look pretty lonely! This is where the realistic scale of the solar system starts becoming an issue. Mercury is small, and compared to the radius of the sun it's still a long way away. So, we'll add a global scaling factor to the radius (to increase it) and the position (to decrease it). This scaling factor will be constant so the relative position of the planets will still be realistic. We'll tweak this value until we are comfortable with how visible our objects are within the scene.

const planetRadiusScale = 1e2;
const planetOrbitScale = 1e-1;
// ...in buildPlanet():
const geometry = new THREE.SphereGeometry(planetRadiusScale * radius, 32, 32);
// ...
mesh.position.set(
    planetOrbitScale * initialPosition.x,
    planetOrbitScale * initialPosition.y,
    planetOrbitScale * initialPosition.z
);

You should now be able to appreciate our Mercury much better!

MOAR PLANETZ

We now have a reasonably-reusable planetary factory. Let's copy and paste spam a few times to finish fleshing out the "inner" solar system. We'll pull our key values from a combination of Wikipedia and our eyeballs' best guess of some approximate color.

// ...in onWindowLoad():
window.app.scene.add(buildPlanet(2.4e3, new THREE.Vector3(57.91e6, 0, 0), 2 * Math.PI / 86400 / 87.9691, 0x666666));
window.app.scene.add(buildPlanet(6.051e3, new THREE.Vector3(108.21e6, 0, 0), 2 * Math.PI / 86400 / 224.701, 0xaaaa77));
window.app.scene.add(buildPlanet(6.3781e3, new THREE.Vector3(1.49898023e8, 0, 0), 2 * Math.PI / 86400 / 365.256, 0x33bb33));
window.app.scene.add(buildPlanet(3.389e3, new THREE.Vector3(2.27939366e8, 0, 0), 2 * Math.PI / 86400 / 686.980, 0xbb3333));

Hey! Not bad. It's worth putting a little effort into reusable code, isn't it?

But this is still something of a mess. We will have a need to reuse this data, so we shouldn't copy-paste "magic values" like these. Let's pretend the planet data is instead coming from a database somewhere. We'll mock this up by creating a global array of objects that are procedurally parsed to extract our planet models. We'll add some annotations for units while we're at it, as well as a "name" field that we can use later to correlate planets, objects, data, and markup entries.

At the top of the module, then, we'll place the following:

const planets = [
    {
        "name": "Mercury",
        "radius_km": 2.4e3,
        "semiMajorAxis_km": 57.91e6,
        "orbitalPeriod_days": 87.9691,
        "approximateColor_hex": 0x666666
    }, {
        "name": "Venus",
        "radius_km": 6.051e3,
        "semiMajorAxis_km": 108.21e6,
        "orbitalPeriod_days": 224.701,
        "approximateColor_hex": 0xaaaa77
    }, {
        "name": "Earth",
        "radius_km": 6.3781e3,
        "semiMajorAxis_km": 1.49898023e8,
        "orbitalPeriod_days": 365.256,
        "approximateColor_hex":  0x33bb33
    }, {
        "name": "Mars",
        "radius_km": 3.389e3,
        "semiMajorAxis_km": 2.27939366e8,
        "orbitalPeriod_days": 686.980,
        "approximateColor_hex":  0xbb3333
    }
];

Now we're ready to iterate through these data items when populating our scene:

// ...in onWindowLoad():
planets.forEach(p => {
    window.app.scene.add(buildPlanet(p.radius_km, new THREE.Vector3(p.semiMajorAxis_km, 0, 0), 2 * Math.PI / 86400 / p.orbitalPeriod_days, p.approximateColor_hex));
});

Adding Some Tracability

Next we'll add some "orbit traces" that illustrate the path each planet will take during one revolution about the sun. Since (for the time being, until we take into account the specific elliptical orbits of each planet) this is just a circle with a known radius. We'll sample that orbit about one revolution in order to construct a series of points, which we'll use to instantiate a line that is then added to the scene.

This involves the creation of a new factory function, but it can reuse the same iteration and planet models as our planet factory. First, let's define the factory function, which only has one parameter for now:

function buildOrbitTrace(radius) {
    const points = [];
    const n = 1e2;
    for (var i = 0; i < (n = 1); i += 1) {
        const ang_rad = 2 * Math.PI * i / n;
        points.push(new THREE.Vector3(
            planetOrbitScale * radius * Math.cos(ang_rad),
            planetOrbitScale * radius * Math.sin(ang_rad),
            planetOrbitScale * 0.0
        ));
    }
    const geometry = new THREE.BufferGeometry().setFromPoints(points);
    const material = new THREE.LineBasicMaterial({
        // line shaders are surprisingly tricky, thank goodness for THREE!
        "color": 0x555555
    });
    return new THREE.Line(geometry, material);
}

Now we'll modify the iteration in our onWindowLoad() function to instantiate orbit traces for each planet:

// ...in onWindowLoad():
planets.forEach(p => {
    window.app.scene.add(buildPlanet(p.radius_km, new THREE.Vector3(p.semiMajorAxis_km, 0, 0), 2 * Math.PI / 86400 / p.orbitalPeriod_days, p.approximateColor_hex));
    window.app.scene.add(buildOrbitTrace(p.semiMajoxAxis_km));
});

Now that we have a more three-dimensional scene, we'll also notice that our axis references are inconsistent. The OrbitControls model assumes y is up, because it looks this up from the default camera frame (LUR, or "look-up-right"). We'll want to adjust this after we initially instantiate the original camera:

// ...in onWindowLoad():
app.camera.position.z = 1e7;
app.camera.up.set(0, 0, 1);

Now if you rotate about the center of our solar system with your mouse, you will notice a much more natural motion that stays fixed relative to the orbital plane. And of course you'll see our orbit traces!

Clicky-Clicky

Now it's time to think about how we want to fold in the markup for the challenge. Let's take a step back and consider the design for a moment. Let's say there will be a dialog that comes up when you click on a planet. That dialog will present the relevant section of markup, associated via the name attribute of the object that has been clicked.

But that means we need to detect and compute clicks. This will be done with a technique known as "raycasting". Imagine a "ray" that is cast out of your eyeball, into the direction of the mouse cursor. This isn't a natural part of the graphics pipeline, where the transforms are largely coded into the GPU and result exclusively in colored pixels.

In order to back out those positions relative to mouse coordinates, we'll need some tools that handle those transforms for us within the application layer, on the CPU. This "raycaster" will take the current camera state (position, orientation, and frustrum properties) and the current mouse position. It will look through the scene graph and compare (sometimes against a specific collision distance) the distance of those node positions from the mathematical ray that this represents.

Within THREE, fortunately, there are some great built-in tools for doing this. We'll need to add two things to our state: the raycaster itself, and some representation (a 2d vector) of the mouse state.

window.app = {
    // ... previous content
    "raycaster": null,
    "mouse_pos": new THREE.Vector2(0, 0)
};

We'll need to subscribe to movement events within the window to update this mouse position. We'll create a new function, onMouseMove(), and use it to add an event listener in our onWindowLoad() initialization after we create the raycaster:

// ...in onWindowLoad():
window.app.raycaster = new THREE.Raycaster();
window.addEventListener("pointermove", onPointerMove);

Now let's create the listener itself. This simply transforms the [0,1] window coordinates into [-1,1] coordinates used by the camera frame. This is a fairly straightforward pair of equations:

function onPointerMove(event) {
    window.app.mouse_pos.x = (event.clientX / window.innerWidth) * 2 - 1;
    window.app.mouse_pos.y = (event.clientY / window.innerHeight) * 2 - 1;
}

Finally, we'll add the raycasting calculation to our rendering pass. Technically (if you recall our "three parts of the game loop" model) this is an internal update that is purely a function of game state. But we'll combine the rendering pass and the update calculation for the time being.

// ...in animate():
window.app.raycaster.setFromCamera(window.app.mouse_pos, window.app.camera):
const intersections = window.app.raycaster.intersectObjects(window.app.scene.children);
if (intersections.length > 0) { console.log(intersections); }

Give it a quick try! That's a pretty neat point to take a break.

What's Next?

What have we accomplished here:

We have a representation of the sun and inner solar system
We have reusable factories for both planets and orbit traces
We have basic raycasting for detecting mouse collisions in real time
We have realistic dimensions (with some scaling) in our solar system frame

But we're not done yet! We still need to present the markup in response to those events, and there's a lot more we can add! So, don't be surprised if there's a Part Two that shows up at some point.

[19/52] Engineering Fundamentals: Statics

Brian Kirkpatrick — Tue, 10 Sep 2024 16:45:58 +0000

Are you excited? I'm excited! Today we're going to dive into our first specific topical area in engineering. Specifically, we'll take a crack at "statics", which is a simple way of restating some basic Newtonian principles around problem solving that grow in the direction of structural engineering.

Statics isn't just about structures, as simple as most examples might look. Rather, this represents a key approach to engineering problems: First, we identify specific equilibrium points in a system. (Once those are defined, we can look at variations about those equilibrium points to learn how they can be stabilized and controlled--but that's later.)

Forces

Recall our vector formulation of Newton's Second Law:

\Sigma \vec{F} = m \frac{d^2}{dt^2} \vec{x}

There is also a corresponding analogue for rotational dynamics, an important topic that we haven't dived into a lot yet--but there's no time like the present!

\Sigma \vec{M} = \Sigma_i \vec{r}_i \times \vec{F}_i

A "moment", in this case, you may recognize as an accumulation of torques. Torque, of course, is simply the leverage of a force about a pivot point, or centroid. We'll dive into these in more detail shortly.

Systems

More importantly, though, is a fundamental assumption of statics (not to mention where "statics" gets its name from): Because the system is static (inertial, or non-accelerating), the forces must sum to zero. The moments, too, must sum to zero. If we repeat these constraints across all three dimensions, we have a set of six equations that can help us solve for the state of complex systems.

Each component of the force sum, of course, must add to zero separately:

ΣiFxi=0\Sigma_i F_{xi} = 0

ΣiFyi=0\Sigma_i F_{yi} = 0

ΣiFzi=0\Sigma_i F_{zi} = 0

Moments are trickier because the cross-product introduces a relationship across dimensions. But, recall from linear algebra that the cross product can be re-expressed as a function of the angle between the two vectors, which we will denote as \theta:

\vec{r} \times \vec{F} = \left| \vec{r} \right| \left| \vec{F} \right| \sin(\theta)

We can use this relationship to expand our moments-must-sum-to-zero into another three specific constraints:

Σi(yFz−zFy)+Σi(Mcos⁡(θx))i=0\Sigma_i (y F_z - z F_y) + \Sigma_i (M \cos(\theta_x))_i = 0

Σi(zFx−xFz)+Σi(Mcos⁡(θy))i=0\Sigma_i (z F_x - x F_z) + \Sigma_i (M \cos(\theta_y))_i = 0

Σi(xFy−yFx)+Σi(Mcos⁡(θz))i=0\Sigma_i (x F_y - y F_x) + \Sigma_i (M \cos(\theta_z))_i = 0

Together, we can use these constraints to encompass a wide variety of systems with the help of just a few tools. In particular, we'll often want to solve for the force or moment about specific elements within a system, like specific cables or specific pins of a truss. These are important ways to characterize the design of a system--but in statics it all comes back to the fact that the system itself is not changing over time.

Centroids

Measuring a force is easy--especially with a known point of contact or point of analysis. Measuring a moment is a little bit trickier because it must be evaluated with respect to some centroid. In the case of a lever, this point is obvious, but in the case of a free body we need a way to compute this "centroid" purely from the geometry of the object.

Fortunately, this is just a mean (or average), and we can express that with THE POWER OF CALCULUS. But, this might look a little different depending on what geometry we are evaluating. Let's consider a linear geometry (like a beam), but to make things interesting let's consider the possibility that it may not have uniform density. Instead, there will be some function of the length along this geometry (typically mass) that we need to integrate infinitesimally-small increments of. Let \vec{c} be the position of the centroid; then,

cx=1L∫xdLc_x = \frac{1}{L} \int x dL

cy=1L∫ydLc_y = \frac{1}{L} \int y dL

cz=1L∫zdLc_z = \frac{1}{L} \int z dL

Typically, this comes down to two steps: First, evaluating the integral along the entire length (e.g., L); Second, evaluating the integral with respect to that length (after all, we are effectively looking for a weighted average.) We'll see exactly how this breaks out in more detail in one of our example problems. But we can evaluate this integral against all forms of geometries, including areas and volumes. For an area:

cx=1A∫xdAc_x = \frac{1}{A} \int x dA

cy=1A∫ydAc_y = \frac{1}{A} \int y dA

cz=1A∫zdAc_z = \frac{1}{A} \int z dA

And for a volume:

cx=1V∫xdVc_x = \frac{1}{V} \int x dV

cy=1V∫ydVc_y = \frac{1}{V} \int y dV

cz=1V∫zdVc_z = \frac{1}{V} \int z dV

Moments of Inertia

Rotational dynamics are funny, no doubt about it. The moment of inertia, particularly important when we begin to consider bending and other torsional loads. In those cases, there is a polar axis about which deformation takes place, and two orthogonal axes (typically x and y). The moment of inertia about each axis is expressed as an integral against the cross-sectional area in question:

Ix=∫x2dAI_x = \int x^2 dA

Iy=∫y2dAI_y = \int y^2 dA

Iz=∫(x2+y2)dAI_z = \int (x^2 + y^2) dA

In reality, while I've presented these as a unified set of relationships above, they are evaluated in different contexts. The x and y moments are specifically deformed when a geometry is "bent" (like a diving board), whereas the polar moment of inertia (e.g., z) is deformed when a geometry is "twisted" about the centroidal axis. We even have a key concept, the "perpendicular axis theorem", that observes these three are basically the sum of one another for mutually perpendicular axes.

Machines

There are common mechanisms for capturing how forces, moments, and boundary conditions interact with one another in the context of simple systems. These systems are useful ways to construct more complicated systems, and if you've looked at any "simple machines" content before, most of them will be instantly recognizable.

We'll visit three specific types of systems and attempt to characterize how forces and moments are related within them.

Beams, Cables, and Pins

Beams and cables are a key element in many static systems. Cables are static in tension (e.g., opposing forces at each end are opposite their displacement from the centroid), whereas beams are also static in compression (e.g., opposing forces at each end can be either in opposition to displacement from the centroid, or both aligned toward the centroid).

Both beams and cables let us assume edges in a structure are in equilibrium for static systems, which lets us focus our analysis on pins (or vertices). At the pins, the forces in equilibrium along each edge can be combined to solve for a static system. This typically lets us then resolve the specific force or moment in each element of the system.

There are typically special "pins" or vertices in a system that have some form of boundary condition. We may assume weight of a system acts primarily upon a single point, for example. We may also have points where we assert there is no displacement or moment. We may also have points where we assume a normal "restorative" force is acting upon the system to maintain a static state. Each of these conditions have different representations and will be encountered in our example problems.

Pulleys

The classic pulley is simply a way to redirect forces. Consider the following diagram:

In this context, the forces are related like so:

F_1 = F_2

There are other variations of pulley systems, however. In another classic example, the motion of the pulley effectively averages the two forces. In the second example from above, the forces are related like so:

F_1 = \frac{1}{2} F_2

Of course, you can combine pulleys one after the other in a "chain" that reduces the force in proportion to the scale of the system. Much like a bicycle chain, you are effectively "trading" displacement for dilution of force. In the third example from above, for n pulleys coupled in this manner, the forces are related like so:

F_1 = \frac{1}{n} F_2

Lastly, you can take advantage of (one might even say "leverage") the difference between a stepped pulley to effectively "differentiate" the displacement resulting from some intermediary force. This effectively leverages the different torques about a shared axis to once again trade displacement dilution for force dilution. In the fourth example from above, given an inner diameter d and an outer diameter D, the forces are related like so:

F_1 = F_2 \frac{1}{2} \left(1 - \frac{d}{D}\right)

Trusses

Combining a number of the above concepts, we can introduce the idea of a "truss". Trusses are complex load-bearing systems that combine beams, cables, and pins, as well as other elements. Trusses effectively form a "graph" of relationships between these elements, and if the constraints equal the degrees of freedom (this should sound familiar by now!) we can "solve" the system by constructing a free body diagram about each pin.

Even if it is overconstrained, we recognize that some elements in this graph may be redundant and can be ignored when evaluating loads. An undersolved system, however is "indeterminate". We can combine our degrees of freedom and constraints observations to determine when a static system is determinate:

n_{members} = 2 n_{joints} - 3

But remember, this is a static system. OFTEN WE ASSUME NO TORSION OR DEFORMATION OF THE SYSTEM, even (or especially) if it is determinate.

Friction

We've been dealing with ideal systems and ignoring friction, because FRICTION EXISTS IN OPPOSITION TO MOTION--and there shouldn't be motion because we are considering STATIC systems. However, friction is a surprisingly complex concept and there are a number of ways to model it. Doing so lets us expand our analysis to several additional interesting machines.

Models of Friction

Classically, the force of friction is proportional to a normal force by some constant frictional coefficient, \mu. Because the normal (or restorative) force enforces a static condition normal to a boundary (like a wall, ramp, or other surface), the motion against which the force of friction opposes is typically orthogonal to this vector--but we can express a relationship of magnitudes:

F_{friction} = \mu N

In reality, the frictional coefficient is not constant. It is the result of a complex interchange of microscopic irregularities between two adjoining surfaces. This means we must recognize different dynamic conditions where these friction must be modeled with different coefficients. This is typically expressed as a series of inequalities.

\leq \mu_{static} N

In the above condition, there is NO SLIP if the applied force is less than a product of a "static" frictional coefficient. As we approach this threshold, we cannot guarantee slip will occur--so we instead say it is "impending":

\mu_{static} N

Once slip occurs, we recognize that friction still exists--but is less in magnitude between moving boundaries. We model this with a different ("kinetic") frictional coefficient, which is typically smaller:

\mu_{kinetic} N

Let's assume a single, constant frictional coefficient for our subsequent analysis of statics, however.

Belts

A belt is a cable-like element partially wrapped around a pulley. The friction at the boundary between these two machines "transfers" some force between the two. Consider the following diagram:

The forces in this machine are related like so, where \theta is the angle about the pulley for which the belt is in contact.

F_1 = F_2 \exp{\mu \theta}

The resulting torque about the axes of this pulley is simply the difference between these two forces about the radius r of the pulley---simple leverage, really.

T = (F_1 - F_2) r

Ramps and Screws

Consider a free-body diagram of a classic "block-on-ramp" scenario:

Assuming a static system, we can observe (from a frame aligned with the angle of the ramp) a relationship between the weight and applied force:

F_1 = F_2 \sin(\phi)

A screw is a simple machine that extends this ramp principle about an axis. The screw bears some axial load (P) by resisting a moment (M) with this force of friction.

We can express this relationship as a function of the radius r of the screw, the pitch angle of the threads \alpha, and a "thread friction angle" that expresses an opposing or augmenting (e.g., positive or negative offset) frictional force as a trigonometric function of the frictional coefficient:

\tan\left(\alpha \pm \tan^{-1}(\mu) \right)

Screws are, of course, incredibly useful. You might not see them when evaluating a truss, for example, but you WILL see them when evaluating structural designs and forces at pins (especially for failure modes).

Example Problems

That's it for the core principles. But if your experiences are like mine, actually solving a statics problem can seem like a little bit of black magic until you've walked through the common techniques a couple of times. So, let's tackle a healthy distribution of examples together.

Problem 1

Consider a lever whose fulcrum is 40% of the distance from the left-hand side towards the right-hand side. Assume the mass of the lever is 100 kg. If Random Person, who weighs 50 kg, starts walking from the left to the right, at what point will the lever be balanced?

Consider the following diagram:

When the lever is balanced, we have a statics problem because all forces and moments are zero. In particular, we need to assert two opposing moments are equal to each other. Choosing the fulcrum as the centroid of the system, we solve for two moments (with sign conventions such that they sum to zero, measuring displacement from the fulcrum point):

0 = M_{weight} + M_{person}

The moment of the lever's weight can be computed once we assume that the lever is uniformly dense. This lets us place the weight vector at the halfway point, or at x = 0.1 L to the right of the fulcrum. If positive moment is right-handed about the fulcrum, this means the moment due to lever weight is therefore negative:

M_{weight} = - F_{weight} x_{weight} = - (100 g) (0.1 L) = - 10 g L

The opposing moment will be displaced in the opposite direction, given some displacement from the centroid x, and will be positive in our right-handed reference about the fulcrum:

M_{person} = F_{person} x_{person} = (50 g) (x L) = 50 g L x

We can, of course, now solve for the displacement x:

\Rightarrow x = 0.2

In other words, the Random Person will be to the left of the fulcrum by 20% the length of the lever.

Problem 2

Consider a pole supporting a beam that bears two signal lights over a pair of train tracks. The first signal is half the length of the beam away from the pin, and the second signal is the full length of the beam away from the pin. What is the moment that must be supported at the pin? Assume both signals have a weight of 70 kg, that the beam is 10 m long, and that gravitational acceleration is 9.81 m/s^2.

Consider the following diagram:

Let's first define our frame:

Horizontal displacement will be 0 at the pin and positive towards the right
Moment will be considered positive if right-handed about the pin
We will denote the moment from the first signal's weight as M_1, and the moment from the second signals' weight as M_2

The net moment about the pin is the sum of the moments from each signal. (We ignore weight as the mass of the beam was not included in the problem statement.) For a statics problem, there is some restoring moment about the pin from the integrity of the structure; this lets us assert a net moment of 0 and solve for the restoring moment accordingly (which, you will note, must be positive).

\Sigma M = 0 = M_R + M_1 + M_2

Replace the signal moments with the signed product of their displacement from the pin and their weight.

M_1 = - F_{g1} x_1 = -70 * 9.81 * 0.5 * 10 = -301

M_2 = - F_{g2} x_2 = -70 * 9.81 * 1.0 * 10 = -602

We can now substitute into our static sum and solve for the restoring moment:

M_R - 301 - 602 \Rightarrow M_R = 903

Problem 3

Consider the tire of a truck driving through mud. The powertrain is applying a moment of 250 [kg m^2/s^2] against the mud through a tire with a radius of 0.4 [m], and this tire supports a quarter of the truck's weight (the truck has a mass of 5000 [kg]). Assume gravitational acceleration is 9.81 [m/s^2]. What is the frictional coefficient if the slip condition is met under these conditions?

Consider the following diagram:

The force applied by the tire against the ground can be computed from the moment and tire radius:

F_M = \frac{1}{r} M = \frac{1}{0.4} 250 = 625

The slip condition will be met when this applied force is equal to friction's opposition to motion, which is a function of the frictional coefficient and the normal force resulting from the truck's weight:

F_f = \mu m g = \mu \frac{1}{4} 5000 9.81 = 12300 \mu

At the slip condition, these expressions will be equal and we can therefore solve for the frictional coefficient \mu:

F_f = F_m \Rightarrow 625 = 12300 \mu \Rightarrow \mu = 0.051

Problem 4

Consider the area defined in an x/y plane by the x-axis boundary, y-axis boundary, and the curve y=\cos(x). This area is continuous for the domain 0<=x<=\pi/2. Compute the x and y coordinates of this area's centroid.

Consider the following diagram.

Recall the formulation for centroids of a two-dimensional area:

c_x = \frac{1}{A} \int x dA

c_y = \frac{1}{A} \int y dA

First, let us compute the area A. This is easily done by integrating along either axis--we will do so along the x-axis, which is considerably easier:

\int_0^{\pi/2} \cos(x) dx = \sin(x) |_0^{\pi/2} = 1 - 0 = 1

We can differentiate the function for area to obtain both dx and dy expressions. Let's start with the x-axis:

\int_0^x \cos(x) dx \Rightarrow \frac{dA}{dx} = \cos(x)

Now we can solve for the x coordinate of the centroid:

c_x = \int_0^{\pi/2} x dA = \int_0^{\pi/2} x \cos(x) dx = \left(\cos(x) + x \sin(x)\right)|_0^{\pi/2} = \frac{\pi}{2} - 1

Now let's differentiate with respect to the y-axis, where we must invert the curve:

\int_0^y \cos^{-1}(y) dy \Rightarrow \frac{dA}{dy} = \frac{dA}{dy} = \cos^{-1}(y)

We can now solve for the y coordinate of the centroid.

c_y = \int_0^1 y dA = \int_0^1 y \cos^{-1}(y) dy

This isn't straightforward, though, so I would encourage you to verify via Wolfram Alpha (or your nearest friendly mathematics major).

c_y = \left(\frac{1}{4}\left[ -\sqrt{1-y^2}y + 2y^2\cos^{-1}(y) + \sin^{-1}(y)\right]\right)|_0^1

c_y = \left(\frac{1}{2}\cos^{-1}(1) + \frac{1}{4}\sin^{-1}(1)\right) - \left(\frac{1}{4}\sin^{-1}(0)\right) = \frac{\pi}{8}

Problem 5

Consider the following truss configuration. If the weight of the truss is modeled by a force of 10,000 N down from the bottom-middle joint, and the length of each horizontal beam is 1 m (with an angle above the horizontal of 45 deg for all diagonal beams at each joint), determine the following:

The restorative force at the joint R_1
The restorative force at the joint R_2
The force (tension or compression) borne by the beam AB
The force (tension or compression) borne by the beam DE

The restorative forces are most easily computed by calculation of the moment about point A. Note we assume a frame at each point--for example, right-handed moments about A are positive, and vice-versa. If the values turn out negative, we will know we need to flip the direction.

M@A=0=2∗R2−1∗W⇒R2=12W=5,0000NM_{@A} = 0 = 2 * R_2 - 1 * W \Rightarrow R_2 = \frac{1}{2} W = 5,0000 N

You can repeat this equation for joint C, but it should not surprise you to learn, for a symmetrical problem, that the restorative forces are identical at both ends.

Trusses are best solved by expressing x and y forces summed to zero (remember, we are in statics problems) at each point, beginning with points with external forces (like weight and restoratives) and "marching" towards the desired unknowns until the constraints let you back out values for each force.

As mentioned above, we will assume a frame at each point and correct when negative signs indicate we assumed incorrectly. Forces must be equal and opposite for each beam, so using a consistent frame and nomenclature will help keep us from messing up too badly. We'll assume positive x to the right, positive y up, and assume each force acts towards the point unless/until we know otherwise.

Let's start by summing forces at point A. Here is the free-body diagram to consider about that joint.

First, let's sum forces to zero about the x axis:

Fx@A=0=−FAB−FADcos⁡(π4)F_{x@A} = 0 = -F_{AB} - F_{AD} \cos(\frac{\pi}{4})

Next, let's sum forces to zero about the y axis:

Fy@A=0=R1−FADsin⁡(π2)F_{y@A} = 0 = R_1 - F_{AD} \sin(\frac{\pi}{2})

Since we know R_1, we know F_{AD}. And since we know F_{AD}, we know F_{AB}. So far so good! In fact, F_{AB} was one of our unknowns to solve for, let's compute that now.

FAD=R122=25000=7071NF_{AD} = R_1 \frac{2}{\sqrt{2}} = \sqrt{2} 5000 = 7071 N

FAB=−FADcos⁡(π4)=−707122=5000NF_{AB} = -F_{AD} \cos(\frac{\pi}{4}) = -7071 \frac{\sqrt{2}}{2} = 5000 N

Let's proceed to summing forces at point D. Here is the free-body diagram to consider about that joint.

First, let's sum forces to zero about the x axis:

Fx@D=0=FADcos⁡(π4)−FBDcos⁡(π4)−FDEF_{x@D} = 0 = F_{AD} \cos(\frac{\pi}{4}) - F_{BD} \cos(\frac{\pi}{4}) - F_{DE}

Next, let's sum forces to zero about the y axis:

Fy@D=0=FADsin⁡(π4)+FBDsin⁡(π4)F_{y@D} = 0 = F_{AD} \sin(\frac{\pi}{4}) + F_{BD} \sin(\frac{\pi}{4})

Since we know F_{AD}, we can solve for F_{BD}. And since we can determine both of those, we can solve for F_{DE}, which was our final objective. Let's do it!

F_{BD} = -F_{AD} = -7071 N

Finally we can solve for F_{DE}.

FDE=22FAD−22FBD⇒FDE=22(7071−−7071)=10,000NF_{DE} = \frac{\sqrt{2}}{2} F_{AD} - \frac{\sqrt{2}}{2} F_{BD} \Rightarrow F_{DE} = \frac{\sqrt{2}}{2} (7071 - -7071) = 10,000 N

Cool beans!

Summary

In statics, we use a combination of linear and rotational stasis (forces and moments summing to zero) to impose constraints on a system of variables. Combined with boundary conditions like weight, friction (a product of normal force and a frictional coefficient), and restorative forces, this lets us solve a system--that is, determine forces and moments throughout each element in a more complicated combination.

Other important topics include the concept of machines, in which forces and moments can be combined and transformed in specific ways. Simple machines like these can be aggregated in arrangements that let us manipulated the behavior of a system.

Lastly, we considered several implications of friction with respect to these machines and used trusses as an example of complicated systems.

[18/52] Engineering Fundamentals: Teh Maths

Brian Kirkpatrick — Mon, 26 Aug 2024 21:51:35 +0000

For a survey course, we try and look at just enough detail to "wet your whistle". But today we're going to try and cover the fundamental math concepts in engineering--which is very ambitious. So, this will be a little longer than usual! But we need to have the tools to wrap our heads around complex systems in engineering fields, so this is pretty important.

More to the point, though, with the right tools we can start to identify and enjoy how concepts are unified across all sorts of different problems in engineering. These insights can help us map and solve many interesting challenges across many different domains! But disclaimer: it's been a while since I had to memorize my inverse Laplace transforms and other tricks, so some of this will be a little out of practice for me. Let's (re)learn together!

We're going to break out three specific topic areas in math, recognizing there are many other categories of math (like probability and statistics) that are highly relevant to engineering but aren't part of what we're covering. Specifically, we'll try and break key math concepts into three areas: ideas from Calculus, ideas from Differential Equations, and ideas from Linear Algebra.

Calculus

Let's take a step back and think about "what calculus is". Calculus is what happens when you take infinitely large numbers of infinitely small samples, to the point where we can make continuous analysis of rates of change, and other related topics. In particular, it helps to have a solid grasp of how you move between different "levels" of differentiation ("up" and "down").

Consider Newton's Second Law, which we'll express like so:

\Sigma \vec{F} = m \frac{d^2}{dt^2} \vec{x}

(As an aside: I'll try and stick to LaTeX-style equation markup, which is not consistently interpreted by different document systems. Hopefully just including the raw markup as source will be somewhat-legible!)

This is a little bit different than the traditional F=ma, but there are some good reasons:

We're spatial, so we're breaking both forces and "state" across components in a vector space
We're specifically not looking at acceleration per se, but rather the second derivative of some state vector (which in this case is position). This idea of a "state" vector is CRITICAL.

A "solution" for an engineering problem like this typically goes from some specification of a problem (like a system of related elements, such as springs and masses and dampers) that we use to construct a differential equation using that state and its rates of change. We'll typically be trying to solve for some expression of that state as a function of time.

Consider a very traditional mass-damper-spring system that looks something like this:

We can use our version of Newton's Second Law to construct an equation of motion:

\ddot{x} = -k x - c \dot{x} - m g

We typically formulate this expression in such a way where the highest-order derivative term is on the left-hand side. If we don't have a damper or gravity, of course, this simplifies to a basic spring example by solving for that second derivative:

\ddot{x} = -\frac{k}{m} x

So, we have an expression that defines the rate of change of a state (on the left-hand side) as a function of that state on the right-hand side. This is an equation of motion--in this case, we have a second-order equation of motion. And if we added back in the damping term, it might look something like this:

\ddot{x} = -\frac{k}{m} x - \frac{c}{m} \dot{x}

It's still a second-order equation of motion, though--but now we have a way of "bleeding" energy from the system. With this middle-order term, the system can now lose energy and therefore we're not stuck with a boring (and unrealistic) infinite-oscillation anymore. But this has strong analogies across other systems, as well. Let's consider a traditional "RLC" system in electrical engineering:

In this case, we use the sum of the voltages to express the voltage of the system as a whole. For each element, we'll use current as state since this is how (in a series) the elements can be combined. Each element has a unique expression of its voltage drop as a function of this state, and combining them gives us something like this:

V_{in} = R I + L \dot{I} + \frac{1}{C} \int I

Of course in this case we have an integral term because the "underlying" state is charge (current being motion of charge over time). We can take the derivative of this equation to have a more traditional expression where the second-order derivative can be moved to the left-hand side:

\ddot{I} = \frac{\dot{V}_{in}}{L} - \frac{R}{L} \dot{I} - \frac{1}{L C} I

And we're back where we started with our spring-mass-damper system. Pretty neat! In fact, you'll see strong analogies across many different engineering fields:

LINEAR KINEMATICS: state is F and \Delta{}x, element properties are k, c, and m
ELECTRONICS: state is V and I, element properties are R, C, and L
HYDRAULICS: state is Q and P, element properties are (unconventionally) W, q, and s
ROTATIONAL KINEMATICS: state is torque and relative angular displacement...
PNEUMATICS: practically identical to hydraulics, with some variations for compressability
THERMAL: ...

...etc.

It becomes obvious partway through that we have a couple of common threads here: There are typically a combination of two state variables (sometimes called "across" and "through" variables, depending on how they are combined and/or conserved by different elements in the system) and a few parameters that are used to characterize the behavior of specific elements within that system with respect to the state variables.

Those variables can be used to construct an EQUATION OF STATE (a generalized term for "equation of motion", since we aren't always just interested in motionn) by combining terms for different systems, at which point we can solve for an expression of state as a function of time using a variety of calculus tools that are probably familiar to you.

Differential Equations

It will come as no surprise to learn that we were constructing (as we put together those equations of state) were differential equations. There is some expression of the rate of change of state that is a function of the state itself.

But, very rarely can we just outright solve for the desired expression of that state as a function of time. Typical tricks from calculus, like integration by parts, are of limited utility and don't always work across complex varieties of systems. Instead, we need a reliable way to refactor our perspective of these equations.

This is a little bit like a path-finding problem. It's hard to go directly from the "equations of state" to the "function of time" we need to solve the system. Often in engineering (or any other problem space, for that matter) it helps to change the "basis" in which we are defining our problem. Consider orbital mechanics, where even for a simple restricted two-body problem we still have a very difficult differential equation:

m_2 \ddot{\vec{r}} = -\frac{G m_1 m_2}{\vec{r}\cdot\vec{r}}

We can still solve for the differential form, especially if we use a "gravitational parameter" to combine the expression \mu = G * m_1:

\ddot{\vec{r}} = -\frac{\mu}{\vec{r}\cdot\vec{r}}

But this is still a difficult formulation of the problem to solve (or integrate) with typical calculus tools! Instead, we can refactor our view of the problem by changing the basis with respect to INVARIANTS inherent to the system. This was the key insight provided by Kepler (for example, constraints of "conic sections" and "constant area in constant time"). Expressing those things mathematically let us change the "basis" of our problem space from six constantly-changing variables (position and velocity in three dimensions) into a space of "classical orbital elements":

a, or semi-major axis, which is a measurement of the scale of the constant ellipse
e, or eccentricity, which is a measurement of the skew of the constant ellipse
i, or inclination, which is a measurement of the orbital plane as constantly tilted from some reference plane
\Omega, or right-ascension of ascending node, which is a measurement of the orbital plane rotated (to align inclination) with respect to some constant reference
\omega, or argument of periapsis, which is a measurement of some rotation within the orbital plane from the ascending node to the vector of "periapsis" (the point of shortest distance from a foci of the ellipse to the orbital trajectory)
\theta, or true anomaly, a measurement of the angle (at any given point in time) within the orbital plane from periapsis to the current location of the satellite in question

With this new "basis", all but one of our state variables are constant! The remaining problem is, to come up with an expression for the only changing variable (anomaly) with respect to time. Everything else becomes a problem of constant geometry and frame transformations. This idea of refactoring the basis of a problem around invariants is an incredibly powerful tool.

In a generic sense, then, we can utilize a tool called "Laplace transforms" that let us linearize the concept of rates of change from calculus. Moving our equations of state into laplacian space lets us take advantage of key properties like:

\frac{d}{dt} \Leftrightarrow s

...and...

\int dt \Leftrightarrow \frac{1}{s}

Let's take a generic second-order differential equation and see how this tool can be used to "solve" for a specific system:

\ddot{x} = a \dot{x} + b x + c

This equation now becomes, using our Laplacian variable s, something like:

s^2 x = s a x + b x + c

And we can now solve for x:

c = (s^2 + a s + b) x

Once we have this expression, we can utilize "inverse Laplace transforms" to reformulate this equation as an expression of our state over time--which is always the end goal. That means, by the way, that it's very much worth memorizing key inverse Laplace transforms (or ILTs). I would break seven key Laplace transforms into three parts:

First, the "identities", beginning with a "unit impulse" in time \delta(t). If we integrated this, we would be dividing by s, and the expression over time would be a unit step. And if we integrated again, we would divide by s once more and, in the time domain, have a basic ramp function:.
Second, there are some modestly-useful exponential forms you can memorize as well. These typically show up when partial fraction expansion is used to break up more complicated systems.
BUT, the two that you REALLY should memorize are the second-order expressions, which you will use most often

Using these fundamental ILTs, we can start combining them to construct more complex expressions of state and using them to characterize the dynamic behavior of the system. For example, we can map specific system element expressions (like spring, mass, and damper coefficients) to things like "settling time" and "natural frequency", which are natural consequences of these second-order ILTs.

In fact, most systems you deal with will decompose into sums of specific ILTs. This lets us come to some very interesting conclusions about system dynamics, like determining which terms will "dominate" the response of the system (by comparing exponentials) and how energy is managed (by comparing oscillation frequencies). Being able to characterize systems using combinations of these ILTs is a very powerful tool!

Example

Consider our original "mass-spring-damper" system. We had some expression of the equation of state as a sum of forces, which we can adjust to come up with an expression in Laplacian space:

m s^2 x = -m g - c s x - k x

Solving for x in this case gives:

\frac{-m g}{m s^2 + c s + k}

You'll notice, as we simplify this expression, a lot of the key parameters end up being ratios between different elements in the system:

\frac{-g}{s^2 + \frac{c}{m} s + \frac{k}{m}}

We can now transform this expression back into the time domain by mapping against the corresponding ILT, in which case we find specific expressions for \alpha and \beta:

\alpha = \frac{c}{2 m}

...and...

\beta = \sqrt{\frac{k}{m} - \frac{c^2}{4 m}}

So, solving for these values, we can come up with an expression for the natural frequency of this system's response and other interesting properties. We can even see how we could start designing a system like this by choosing specific values for things like k and c to realize specific dynamic properties of the system.

For example, we may have requirements that specify the settling time of the system should be "no less than 50% in some time t"--in which case we can now back out what the damping coefficient (for example) should be. Or, in electrical engineering, we may now know what resistors (with what resistances) should be chosen for specific elements. Etc.

In general, let's summarize our approach:

We have some knowledge of system dynamics, which let us express the rates of change of state as a function of the state
We can use this to solve for an expression of the state over time
That expression will have key parameters that map back to the attributes of specific system elements
We'll be able to express the dynamics of the system as a function of these elements
Given requirements or constraints we impose upon that system, we can "design" the system by choosing values of those attributes to satisfy the requirements

Linear Algebra

Let's reconsider the "state space representation" from one of our previous articles.

Recall we had: 1) some "plant" model of a system being controlled; 2) some observable output of that system as recorded by "sensor" systems; 3) a "controller" that would use these sensor measurements to adjust the command or control; 4) some set of "actuators" that could translate these commands into a change to the input or state of the controlled system ("plant").

In this scenario, we could construct a model of the system dynamics that looked something like this:

\dot{\vec{x}} = A(\vec{x}) + B(\vec{u})

If this is a "linear, time-invariant" system (or LTI), we know that A and B are going to be constant-value matrices. And we have a similar expression for the output of the system:

\vec{y} = C \vec{x} + D \vec{u}

We can utilize our Laplacian variable to substitute the left-hand side of the first equation and solve:

\vec{x} = A \vec{x} + B \vec{u}

And now we can use linear algebra to solve this expression for the state vector:

\vec{x} = (s I - A)^{-1} B \vec{u}

With some knowledge of linear algebra, we know that the middle part of the expression is crazy. Basic linear algebra properties like the determinant, eigen-vectors, and eigen-values of this matrix expression will tell us A LOT about the controllability of the system, and other key system dynamics.

s I - A)^{-1}

"But wait", you might say, "this is still first-order, and first-order is boring!"

There's a fun trick we can do that ensures these conclusions (and properties) are applicable to any system of any order. Keep in mind that \vec{x} is an unconstrained number of dimensions. Let's say this captures a position in space:

\vec{x} = \left[r_x, r_y, r_z\right]

That means the derivative of state would be:

\dot{\vec{x}} = \left[v_x, v_y, v_z\right]

But, if we expand our definition of state to include velocity AND position, then our state vector becomes:

\vec{x} = \left[r_x, r_y, r_z, v_x, v_y, v_z\right]

In this case, we now have the following derivative of state:

\dot{\vec{x}} = \left[v_x, v_y, v_z, a_x, a_y, a_z\right]

Now we can recognize how the matrix A will change to capture system dynamics about this new state vector: The "upper-left" will just be zero, the "upper-right" will be an identity (capturing v_x=v_x, etc.), and the key dynamics of the system will be in the "lower-right" (and "lower-left") of the matrix. Among other things, this lets us apply a few other helpful linear algebra insights into characterize the different dynamics. And of course this lets us extend to nth-order system dynamics.

We can take this a step further as well. There are algebraic properties of the expression (s * I - A)^-1 that we will be interested in:

Is it invertible?
What is the determinant? We call the expression for the determinant of this matrix the "characteristic equation" of the system.
From the characteristic equation, which will be something like s^2+a*s+b, we can derive many interesting conclusions by looking at the roots of this expression, including a comparison of the "inner" term of the quadratic (a^2 - 4 * b), which may let us characterize the hyperbolic, parabolic, or elliptic nature of the system.
"Eigenstuff" and roots will let us characterize stability (particularly in the case of complex roots) and how controls can be applied to "move" roots around complex space to influence the controlled response of the system

When we had a Laplacian expression for the system dynamics, we had what is sometimes called a "transfer function", or H(s), that captured the "output" over the "input" of the system.

\frac{y}{u} = \frac{1}{s^2 + a s + b} = H(s)

A transfer function lets us multiply this expression by some input to realize some output. If we solve for this transfer function in our state space model using linear algebra tools, we might have some transpose expression based around the system matrices. An expression of this form lets us solve for "optimal" control coefficients (for example, with linear feedback constants \vec{u} = -K * \vec{x}). We can feed this into a quadratic formulation with respect to some cost function J(\vec{x}), and come up with algebraic Riccati equations for non-linear optimization. And many, many other tricks!

Conclusion and Summary

From Calculus, we came away realizing that:

EQUATIONS OF STATE let us express system dynamics as a combination of elements that define a rate of change with respect to system elements and the state of that system
ANALOGOUS SYSTEMS let us apply common solving techniques across diverse problem sets, from mechanics to electronics, using tools like Laplace transforms

From Differential Equations, we came away realizing that:

INVARIANTS and TRANSFORMS let us reformulate complex system dynamics expressions (including equations of state) in ways that we can solve for an expression of state over time
INVERSE LAPLACE TRANSFORMS let us algebraically "solve" for an expression of the system that we can capture in specific terms
Those terms can tell us interesting things about SYSTEM DYNAMICS PROPERTIES, like natural frequency and settling time, as a function of the properties of specific system elements

From Linear Algebra, we came away realizing that:

STATE SPACE representations let us capture system dynamics in multi-order system dynamics matrices
LINEAR, TIME-INVARIANT systems have constant-value matrices that let us construct linear algebra expressions across multiple degrees of state
ALGEBRAIC PROPERTIES of those constant-value matrices can give us key insights into system dynamics, including stability and controllability

WHEW! This was a lot. But it's incredibly juicy stuff, and I hope you enjoyed it as much as I did.

[17/52] CloudInit, DigitalOcean and Terraform (a minecraft adventure)

Brian Kirkpatrick — Fri, 16 Aug 2024 05:08:09 +0000

Software time today, baby! And it's something kind of fun. I swear.

There's a fun project I recently got working that combines a couple of different and interesting technologies, including one of my favorite cloud providers (DigitalOcean); a great way to use their resources (Terraform); and a way to procedurally configure the virtual machines they host (cloud-init). We'll use these technologies to spin up our own Minecraft server!

In the past, I've done this with Docker on my desktop, but this comes with a lot of disadvantages. Persistence is something of a bear, and there's a lot of networking configurations (like port forwarding and open firewall rules through your residential ISP) that aren't ideal, especially if you want other people outside your home to be able to play with your family.

The Fundamental Element

We could, theoretically, go full-up Kubernetes or deploy a container on the cloud provider too. But the fundamental "unit" of most cloud providers is instead a virtual machine of some kind. In the case of DigitalOcean, those VMs are called "droplets". So, let's start by creating a "droplet" specification in Terraform.

We're going to use the DigitalOcean Terraform provider here. This is one of my favorite things about DigitalOcean, by the way--the Terraform provider is well-documented and punches WAY above its weight. It's Azure-levels of quality (way above AWS), but without MSFT.

If you look at the digitalocean_droplet resource documentation, you'll see it's pretty easy to just define one and spin it up. One thing you will want to consult, though, are the "slugs" used for key reference labels (like available images and VM sizes). But this is enough for us to define our "core" resource, the VM/droplet, in our first Terraform file, dodroplet.tf:

resource "digitalocean_droplet" "dodroplet" {
  image     = "ubuntu-22-04-x64"
  name      = "dodroplet"
  region    = var.DO_REGION
  size      = "s-4vcpu-8gb"
  ssh_keys  = [digitalocean_ssh_key.dosshkey.id]
  user_data = data.template_file.user_data_yaml.rendered
}

Some observations:

We're going to pass in the "region" as a variable, so anyone deploying from this Terraform specification can chose exactly where it will spin up
There are a few references, which we haven't defined yet, to resources like an SSH key and a user_data field. The user_data field is particularly interesting, because this is how we'll pass in our "cloud-init" configuration (more on that later).
We're using a basic 4-cpu, 8gb-RAM image here; it's not the cheapest one, but only costs about $0.07/hour; this comes out to about $50/month, which is comparable to another virtual private server I have on a different provider.
Those specs might actually be overkill for our Minecraft server! We could probably get away with a cheaper one if we had to.

Some more words about "cloud-init" and the user_data field: For now, know that "cloud-init" is a way to provide a .YAML-like specification for how a VM should be configured as it boots. Much like Ansible, "cloud-init" can define specific "playbooks" or blocks of properties and behaviors--like what packages need to be installed. In this case, we're saying this content will come from a template that we'll procedurally "render" during deployment, when Terraform interpolates specific values.

Project Namespaces

We'll add a digitalocean_project resource next. This will help us group our resources together into a logical namespace. A project gives us a nice way to organize related resources and, for ease of cost control purposes, lets us delete everything simply by getting rid of the project when we're done. Here's the contents of doproject.tf:

resource "digitalocean_project" "doproject" {
  name        = "domacs"
  description = "Namespace for encapsulation of cloud resources"
  purpose     = "Demonstration"
  environment = "Development"

  resources = [
    digitalocean_droplet.dodroplet.urn,
    digitalocean_domain.dodomain.urn,
    digitalocean_volume.dovolume.urn
  ]
}

(Note we include the droplet, a domain, and a volume; we'll define the other, non-droplet resources in just a moment.)

Variables and Inputs

If you signed up for DigitalOcean, you've seen the control panel from which you can monitor your resources. Use the "API" section of this page to generate a new token. This token is sensitive, but Terraform will need it in order to have the authority to spin up your resources. Here's a screenshot of what to look for.

We'll pass this token in as a variable, or input. Create a variables.tf file and define what these inputs should be like:

variable "DO_TOKEN" {
  type        = string
  description = "API token for deployment of DigitalOcean resources"
}

variable "DOMAIN_NAME" {
  type        = string
  description = "Managed domain name (should point to DigitalOcean NS records) used by the VM"
}

variable "ADMIN_USER" {
  type        = string
  description = "Name of user who will initially be able to connect to the server (e.g., before other whitelist names are added)"
}

variable "ADMIN_UUID" {
  type        = string
  description = "UUID of user who will initially be able to connect to the server (see https://mcuuid.net for easy lookup)"
}

variable "DO_REGION" {
  type        = string
  description = "DigitalOcean region into which resources will be deployed"
}

There are several ways to pass in values for these variables. You can create a terraform.tfvars file that defines basic key="string-value" mappings line-by-line (like DO_REGION="sfo3"), if you're okay with those values touching disk.

You can also define an environmental variable that beings with TF_VAR_, followed by the name of the Terraform variable. This is a great trick--the sensitive values never touch disk and can be automatically mapped from things like CI runner tokens. If you take the former approach, though, make sure you add *.tfvars to your .gitignore file to make sure sensitive values aren't added to version control!

SSH

Let's say we just want to spin up the droplet and start inspecting it. We don't want to hard-code user credentials as part of the VM specification, so we'll set up SSH instead. To do this, we'll take advantage of a neat provider built into Terraform to define a private key, within a tlskey.tf file with the following contents:

resource "tls_private_key" "tlskey" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

Once we've defined the TLS private key, we can use this to define the SSH key resource that will be passed into our droplet (as you may have already noticed from the dodroplet specification above!).

resource "digitalocean_ssh_key" "dosshkey" {
  name       = "dosshkey"
  public_key = tls_private_key.tlskey.public_key_openssh
}

The VM only needs the public part, so it will know to accept users logging in with that key. The private part we will keep for ourself. Specifically, we'll add the private key to our outputs; create an outputs.tf file and include the following:

output "PRIVATE_SSH_KEY" {
  value     = tls_private_key.tlskey.private_key_pem
  sensitive = true
}

output "VM_IP_ADDR" {
  value = digitalocean_droplet.dodroplet.ipv4_address
}

Once our infrastructure is deployed, we'll be able to call terraform output -raw PRIVATE_SSH_KEY > id_rsa to generate a key file that we can use in conjunction with an ssh command. You'll notice we also want to capture and report the IP address of the VM, so we know where we'll be logging into. (Since the latter value is not sensitive, it will be automatically reported by Terraform directly to the console upon deployment.)

cloud-init

We're going to use a template data to define our "cloud-init" configuration. This will be "interpolated" using specific values we want to write into the "cloud-init" behavior. Create a user_data.yaml.tpl file and populate it with the following; there's a lot going on here, so stick with me and I'll explain it in just a moment:

#cloud-config
package_update: true
package_upgrade: true

packages:
  - openjdk-21-jre-headless
  - screen

write_files:
  - path: ${PERSISTENT_VOLUME_PATH}/start_minecraft.sh
    permissions: '0755'
    content: |
      #!/bin/bash
      cd ${PERSISTENT_VOLUME_PATH}
      java -Xmx1024M -Xms1024M -jar minecraft_server.1.21.1.jar --nogui 
  - path: ${PERSISTENT_VOLUME_PATH}/server.properties
    permission: '0755'
    content: |
      difficulty=normal
      white-list=true
  - path: ${PERSISTENT_VOLUME_PATH}/ops.json
    permission: '0755'
    content: |
      [
        {
          "uuid": "${ADMIN_UUID}",
          "name": "${ADMIN_USER}",
          "level": 4
        }
      ]

runcmd:
  - mkdir -o ${PERSISTENT_VOLUME_PATH}
  - cd ${PERSISTENT_VOLUME_PATH}
  - wget -O minecraft_server.1.21.1.jar https://piston-data.mojang.com/v1/objects/59353fb40c36d304f2035d51e7d6e6baa98dc05c/server.jar
  - echo "eula=true" > ${PERSISTENT_VOLUME_PATH}/eula.txt
  - bash ${PERSISTENT_VOLUME_PATH}/start_minecraft.sh

final_message: "Minecraft server setup complete!"

Let's go block by block:

After the shebang, we define several options that tell "cloud-init" to update the package index and perform any upgrades
In the packages block, we list specific packages we want the VM to install, like the OpenJDK runtime
In the write_files block, we define the contents (and filename, and permissions) of several files we want to write into the filesystem; these are located by the variable ${PERSISTENT_VOLUME_PATH} (which we'll pass in to reference our mounted volume), and include things like the list of "operators" initially authorized to connect to our Minecraft server; the server.properties that will let us customize our Minecraft configuration (like what difficulty exists and whether or not the whitelist is enabled); and a shell script used to launch the Minecraft server by launching the "fat .JAR" with the java command.
In the runcmd block, we define several commands that need to be run when the system is launched. Specifically, we need to (within the context of our persistent volume path) make sure the fat .JAR is downloaded; write out the EULA approval; and run the script we defined in a previous block. (You may want to check the official page to make sure you have the latest URL for this "fat .JAR".)
Finally, in the final_message block, we include a message to verify the "cloud-init" configuration has successfully been applied. This can be useful if you lose track of the startup log messages. Another useful technique is to set a custom environmental variable that you can check from the shell upon boot.

Once this file is defined, we need to tell Terraform this can be interpolated and rendered as a template. Create a data.tf file and add the specification for this resource:

data "template_file" "user_data_yaml" {
  template = file("${path.module}/user_data.yaml.tpl")

  vars = {
    ADMIN_USER = var.ADMIN_USER
    ADMIN_UUID = var.ADMIN_UUID
    PERSISTENT_VOLUME_PATH = "/mnt/${digitalocean_volume.dovolume.name}/minecraft"
  }
}

In this case, we're just passing a few variables into the template interpolation, as well as procedurally constructing the path where the persistent volume will be mounted. (This is done simply by referencing the volume name, which we can do procedurally--isn't Terraform great!?)

Providers

If you've used Terraform before, you might recognize we haven't defined our providers yet. We are using some built-in providers, like template and TLS, but we also need to define the DigitalOcean provider. Add a providers.tf and we'll populate it now:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
  }
}

provider "digitalocean" {
  token = var.DO_TOKEN
}

Note that we just pass through the API token from our variable inputs. Easy!

Volume

We haven't defined our persistent volume yet. This will let us ensure the state of the server is maintained, even if the VM itself gets rebooted. This will take two steps: first, defining the volume, and second, mounting (or mapping) it to the droplet. First, create a dovolume.tf and populate it as follows:

resource "digitalocean_volume" "dovolume" {
  region                  = var.DO_REGION
  name                    = "dovolume"
  size                    = 100
  initial_filesystem_type = "ext4"
  description             = "Persistent storage for DOMACS server configuration and world data"
}

(Note that volumes, like droplets themselves, must be deployed to a a datacenter in a specific region. Since we've defined this value as a Terraform variable, ensuring both of them are co-located is a snap, even if other users deploy their infrastructure to other regions.)

Once we've defined the volume, we're ready to "mount" into the droplet--specifically, we'll need to define an "attachment" resource that tells DigitalOcean that our specific VM should mount that specific volume. Create a domount.tf file to do so:

resource "digitalocean_volume_attachment" "domount" {
  droplet_id = digitalocean_droplet.dodroplet.id
  volume_id  = digitalocean_volume.dovolume.id
}

Domain

We could, within Minecraft, just connect to the VM's IP address--but this is inconvenient and can change if/when the VM reboots. Instead, we'll register a domain name (I have several just lying around from various side projects!) and point it to the DigitalOcean nameservers. Then, create a dodomain.tf file that will define the domain resource that performs the A-record mapping automatically:

resource "digitalocean_domain" "dodomain" {
  name       = var.DOMAIN_NAME
  ip_address = digitalocean_droplet.dodroplet.ipv4_address
}

Graphing

One handy thing you can do to verify your infrastructure is to generate a graph of the relationships Terraform has derived. (If you are on Windows, it may be easier to use WSL for the next command, since you will need a command-line vector graphics tool like GraphViz.) You can pipe the infrastructure specification into dot to generate a shiny SVG file:

terraform graph | dot -Tsvg -o graph.svg

Just don't ask me why it things the volume is related to the "cloud-init" configuration! I think some wires got crossed.

(If this doesn't look visually appealing, there are ways to pass these results through other, shinier tools.)

Finally, Deployment!

We have a full-up infrastructure now that is ready for our Terraform commands! Assuming our variables have been defined, we're ready for the traditional three-step:

terraform init
terraform plan
terraform apply

To verify, you can go to your DigitalOcean control panel and look for two things:

Verify the project, droplet, volume, and domain are all created
Use the built-in console to log into the VM and look for key clues that the "cloud-init" configuration completed, like a ps -e | grep java to view running Java procsses; you can also use the SSH key we set up to do the same thing from your local shell, of course.

And of course, if everything looks great, you can log into your new persistent Minecraft server using the domain name!

Before You Commit

You should make sure your .gitignore file is fully populated before you commit and push your contents. This includes the .terraform/ folder, intermediate lock and state files, and (of course) your .tfvars file where your sensitive secrets are stored.

Conclusion

Spinning up a Minecraft server is a great exercise for learning key cloud technologies. Hopefully you've seen how effective combinations of these technologies (like DigitalOcean, Terraform, and cloud-init) can be used to simplify, automate, and proceduralize how your infrastructure is deployed and orchestrated.

[16/52] Engineering Fundamentals: Units & Measurements

Brian Kirkpatrick — Tue, 13 Aug 2024 22:16:29 +0000

After the introductory knowledge graph and transport theorem, today is the first segment of our engineering fundamentals content. Specifically we're going to talk about units and measurement! For engineers, this can be incredibly useful to wrap your head around. Feel comfortable with managing units and measurements--or it will bite you in the derrier.

Physical Basis

Let's start with the physical basis of what units are. And naturally (sarcasm here), that means starting with relativity and quantum! There are three units that stand heads and tails above the rest, even if we're just focusing on the seven "Base SI" units: length (meters), mass (kilograms), and time (seconds).

Specifically, let's think about "very big" measurements in these units and "very small" measurements.

If you've heard about "Plank units", these represent the fundamental lower limit on what measurements in these dimensions (like Plank length) are useful. At that scale, utility breaks down for engineers--we like things to be deterministic, after all, or they will be impossible to control and design for!

Out at the "very big" end, on the other hand, we need to take relativistic effects into account. This is particularly true for large speeds (rate of length vs. time) and large mass (with implications for spacetime). Engineers may sometimes need to compute relativistic corrections and frames, but not very often.

So, for the most part, engineers will live in this "happy medium" between quantum and relativistic ends of these three dimensions (length, mass, and time). And units we typically use can even be expressed as exponential combinations of each dimension: [m]^a * [kg]^b * [s]^c; for example, speed (in meters per second) might be expressed as a=1, b=0, and c=-1. These exponents are typically integers--but not always.

This has interesting implications for the vector space in which units are embedded. But even avoiding those topics, this can be a very useful way to define and model units in ways that are relevant (and memorable) for engineering purposes. This is true for Base SI as well as non-Base SI and Imperial measurements (though there are some edge cases for sure).

Base SI

There are various "systems" of units different problems may use, even avoiding the conversion issues. "Base SI", for example, may also be referred to as mks for "meters, kilograms, seconds". Other systems you may come across (even while still using Metric) may include cgs (for "centimeters, grams, seconds").

Beyond the "core" three units, base SI also includes "dimensions" for current, temperature (of course), quantities, and "luminous intensity" (more on that last one later). That gives us seven fundamental "dimensions" of Base SI units, from which we can express any other measurements or units as a function of these dimensions.

One of the handy things about Base SI units are, there are specific physical references that define each unit. Many Imperial units (or even other Metric units) were originally defined (and still are!) using some "traditional" reference of (for example) what a "stone" might typically weigh. Base SI units have explicit mapping to (for example) the transition of a particular atomic state:

"The second, symbol s, is the SI unit of time. It is defined by taking the fixed numerical value of the caesium frequency, ∆νCs, the unperturbed ground-state hyperfine transition frequency of the caesium 133 atom, to be 9192631770 when expressed in the unit Hz, which is equal to s−1." The International System of Units

This is certainly less ambiguous than a unit like a pint, as we'll see later!

Operations and Tools

When we talk about calculations with measurements, we understand basic arithmetic operations, but there are additional considerations we need to make when those quantities have units. Let's consider addition.

a + b = c

If a and b have different units, this is not a valid operation! Addition is quite a constrained operation with measurements because the units must be identical, and the units of the result will be identical to the units of the operands.

a [m] + b [m] = c [m]

Multiplication, on the other hand, is considerably more interesting because it lets us combine and permute the units we are using. Specifically, the units of the product are the product of the units. And for our purposes, this is true for division too since it is just the inverse.

a [m] / b [s] = c [m/s]

This can be tricky to handle and model, especially in digital circumstances where the computer may think you are just dealing with numbers. That's why systems like "Pint" (in Python) can be very useful, because they let you extend and constrain the manipulation of specific variables based on what units are assigned to them.

https://pint.readthedocs.io/en/stable/

I also find it's helpful to explicitly label units in your variables when programming engineering calculations. For example, you can use camel-case for the variable name but then indicate units following an underscore. With this approach, integer values can be used to indicate power and the letter p (for "per") to indicate a ratio of "before" divided by "after". speedOfLight_mps, for example, might store the speed of light in meters per second, whereas speedOfLight_miphr might store the speed of light in miles per hour (mph would also be acceptable but introduces some ambiguity; I tend to reserve single-letter abbreviations for Base SI, when possible).

I also find it's help to always store internal values as Base SI units, and convert only at the "edge" (program interface or visual presentation). This helps simplify assumptions and minimize misunderstandings during development activities, as well as making it explicit when defining interface requirements.

Imperial and Other Edge Cases

While Base SI gives us a great tool to define a "core" of unit dimensions that we can use for converting to and from other systems, those other systems can come with a lot of edge cases. Imperial is the easy one to blame of course (and we'll start there)--but there are some other tricky areas, too.

Imperial Length

Length is the least tricky of the core dimensions when it comes to Imperial units. Specifically, I find it helps to remember one specific conversion (inches to centimeters) because it has "lossless" precision, and is therefore a great "intermediate" unit regardless of what your source and destination Imperial and Metric units are.

2.54 [cm] = 1 [in]

We can use this as the basis for precise conversion between many other units, where the conversion constant would be harder to remember or even of limited or approximate ("lossy") precision.

Imperial Weight (pounds issues)

There are several "gotchas" to keep in mind when it comes to Imperial weight measurements. We don't have an easily-rememberable constant conversion between (for example) pounds and kilograms. Instead it is typically approximated (if this is sufficient for your calculations) by 2.2 [lbs] = 1 [kg]. Interestingly, though, there is (as of 1959) an explicit conversion factor--it's just much harder to remember:

0.45359237 [kg] = 1 [lb]

Another problem here is the difference between mass and weight. If I tell you I weigh 200 pounds, that's not a measurement of mass! That's a measurement of weight. Weight is a FORCE--specifically, the force exerted by gravity on a specific mass. Confusingly, in Imperial, both are measured in "pounds"--so we need to be very careful. "Pounds" should always specify when you are referring to "pounds-mass" (in which case there is a conversion to kilograms) or "pounds-force" (in which case you need to convert to Newtons).

Imperial Volume (especially pints)

Volume is surprisingly tricky, even though it's just a "cube" of length. Metric volume, of course, is specifically constrained by 1 [cm^3] = 1 [mL]. (A liter equal in volume to a cube 10 centimeters on each side.) I find it is helpful to use fl oz (or "fluid ounces") as a base reference when working with volume measurements in Imperial, for two reasons:

It has a relatively straightforward approximate conversion to Metric: 1 [fl oz] ~= 30 [mL] (this ratio is exact if we are using "US food labeling fluid ounce" standards, but most other conversions will instead use "Imperial fluid ounces" or "US customary fluid ounces").
It has a relatively straightforward "ladder" of conversions up to other Imperial units of volume, which will be familiar to anyone with a decent amount of cooking experience:

8 [fl oz] = 1 [cup]

4 [cups] = 1 [quart]

4 [quarts] = 1 [gallon]

The tricky outlier here are "pints". Even putting aside the difference between "Imperial fluid ounces" and "US customary fluid ounces" (I used the latter in the above examples), there is no standard definition of a pint. You can typically assume (unless stated otherwise or within a specific alternative context) that a pint is two cups, or 16 fluid ounces, but be warned!

Temperature

Temperature is something of an edge case because it's not a proportional conversion if you are using degrees Celsius or degrees Fahrenheit. That relationship is useful to memorize in and of itself, though:

F = 9/5 * C + 32

But hopefully you don't have to deal with the non-proportional measurements; rather, you can convert (as soon as possible) into degrees Kelvin or (somewhat less common) degrees Renkin. In the Celsius-Kelvin case, it's helpful to memorize the constant offset (0 [deg C] = 273.15 [deg K]). Note that we need to mention what kind of degrees we reference! There are different ways to do this (for example, using the special circle character, or changing the spacing, or leaving out deg altogether), but I find the approach in the above example useful.

Radiometry

...and now we get to the fun one. Remember that funky "luminous intensity" unit in Base SI? There's no easy way to tackle the systems of units used in radiometry. But they're important--particularly in thermal analysis, electro-optics, RF calculations, and infrared systems. (They're even useful in computer graphics calculations--e.g., shader calibration--but that's another conversation for another day!)

There are a couple of things about radiometry that make it the devil when it comes to units.

If you ever hear "visual magnitude", that's a logarithmic magnitude calibrated off of human perception--that is, what your eyes perceive. So when you see "VM", convert out of it as soon as possible!
You'll often see some measurement of luminosity or power emission, but those units are further elaborated when you consider the flux of those measurements through space from a point. This is typically done by normalizing about a solid angle--for example, per "solid radians". In this manner, the measurements (such as power) are consistent regardless of how far away you are from the point.
You will also see these units expressed as a unit per wavelength, because there will be spectral concerns that describe how a measurement might be distributed across different frequencies.

Yeah, it can be a pain. Be very, very careful with units when doing radiometric calculations.

Dimensional Analysis and Examples

Conversions

Let's talk about conversions. We can use our knowledge of unit dimensions and operations to help us solve problems. Let's say we want to convert 100 yards into meters. There will be a "chain" of several conversions that will eventually get us to meters. At each step in the chain, we'll multiple by some fraction that will let us "cancel out" the units from the previous step. We'll use a combination of these steps to then define the arithmetic operation (without mixing up which factors are multiplied or divided!) that lets us get the right answer--and with known precision.

So, to start, remember that it's useful to leverage the lossless precision of the "centimeters/inches" conversion. So let's start by converting yards to inches.

100 [yd] * (36 [in] / 1 [yd])

There's a fraction on the right that we can define by asking ourselves, how many inches are the same as how many yards? It is often helpful to first define the quantity in this factor for the smaller unit, because this is often an integer value and therefor more precise. Often, the other value will be 1, and in this manner you'll know (asking which top is the same as which bottom) what factors you are multiplying or dividing by without getting them inverted by mistake.

Now we can apply our lossless conversion from Imperial into Metric:

100 [yd] * (36 [in] / 1 [yd]) * (2.54 [cm] / 1 [in])

And finally convert within the known Metric prefixes:

100 [yd] * (36 [in] / 1 [yd]) * (2.54 [cm] / 1 [in]) * (1 [m] / 100 [cm])

Now we have a specific arithmetic expression we can evaluate (in this case we even get to cancel out 100 on the top with 100 on the bottom), and verify that the units cancel out to give us our desired answer:

`100 [yd] = 91.44 [m]

What's more, because none of our conversion factors were approximate we know this is a precise conversion.. which is nice.

Expanding and Consolidating Unit Expressions

In our Base SI system, we didn't see any units defined for force, or power, or acceleration. That is, of course, because those units are a combination of the units in our Base IS list.

Let's consider power, which (in Metric) is measured in Watts. What does that mean (say, for a 100 [W] light bulb)? Well, Watts are a measurement of the rate of energy, or joules per second:

[W] = [j/s]

Joules, or energy, are an integrated force--a force applied over some distance. So, a joule is a Newton-meter:

[W] = [j/s] = [N * m / s]

A force is a unit of accelerated mass, and acceleration is the second derivative of displacement. SO we can continue expanding our expression of the unit for power:

[W] = [j/s] = [N * m / s] = [kg * m / s^2 * m / s]

We're in Base SI now--and of course that means we can consolidate units into a specific expression of power:

[W] = [kg * m^2 / s^3]

We can even expand this in terms of our exponential expression, embedded with a logarithmic space of these dimensions:

[W] = [kg]^1 * [m]^2 * [s]^-3

Orbital Period

In orbital mechanics, there is a tricky expression for the period of an orbit, as a function of its orbital properties. It's an angular rate, so we can start with an expression of the period per revolution:

T / (2 * pi)

That expression is equal to the area "swept out" by that rate (thanks, Kepler!), but as a square root of some ratio between the "gravitational parameter" (mu, a property of the central body, or Earth, in units of [m^3/s^2]) and the "semi-major axis" or length of the conic section (a in units of [m]).

So, we can determine what the inside of that square root is suppsed to be by what expression gives us seconds-squared, since a^3/mu has units of [m^3 / (m^3 / s^2)], or [s^2]. In other words, we can always back out the expression for orbital period:

T = 2 * pi * (a^3 / mu)^0.5

Gas Constant

Another fun dimensional example is the gas constant. This is a very important quantity (and it's not just my fluids background talking) that pops up often and in very surprising places (and with several different forms or units).

For example, in Base SI units, the gas constant is:

R = 8.314... [j / deg K / mol]

... which is a funky beast. What sort of units are those!? Other expressions are just as strange:

R = 1.9877... [cal / deg K / mol]

(And note that's calories, not kilo-calories, which is a fun mistake to make when you're trying to think about nutrition labels.)

R = 8.206...e-5 [m^3 * atm / deg K / mol]

Why is this so strange? Because R represents an invariant of specific ratios in which there are very different units. Recall the ideal gas equation:

PV = nRT

So, we know R will always have to be a ratio of, on one side, the product of pressure and volume units, and on the other side, the product of particle count (moles) and temperature units. And it's the PV units that work out to something particularly interesting in Base SI:

[N/m^2 * m^3] = [kg * m / s^2 / m^2 * m^3] = [kg * m^2 / s^2]

Conclusion

As engineers, we don't really have the luxury of saying "I'm only going to work in Metric" (for example). You'll hear horror stories of, for example, Mars probes that crash because the subcontractor used miles while the JPL system was navigating in kilometers, etc..

Instead, at the end of the day, an engineer needs to be FLUENT when it comes to units. You need to know the conversions, and to be able to move around within the space off different unit systems. That's what dimensional analysis gives you--a powerful way to make sure you are holding yourself accountable when it comes to important calculations and approximations.

[15/52] The Three Languages an Engineer Needs To Know (or maybe it's more)

Brian Kirkpatrick — Wed, 07 Aug 2024 23:57:46 +0000

We're going to revert to type, just a little bit, for today. I want to develop an idea that has been developing for a while in software engineering. Specifically, I want to tackle the idea that, in order to be a successful software engineer "YOU NEED TO LEARN LANGUAGE XYZ".

What I've found is, instead, it's less about specific languages and more about specific "slots" in your toolbelt. There are specific roles that different languages will provide, largely categorized by how they are used. So, today we're going to break down the three "levels" of programming languages that you will want to learn and use.

Let's start with a specific example of such a "stack", based on languages I've used a lot of in my own experience.

After a while what you'll realize (regardless of what your selections are) that different languages are good at different things. It can be fun to flame this-language-vs-that-language, etc. But there may be different trades with a new project that go into your decision of what language to use. C++ gets a lot of hate! Python gets a lot of hate! Javascript gets a lot of hate! But regardless, there are specific situations where you would choose each of these languages, and there are different roles for which they will be better suited.

The Three Layers

One of the most interesting books I've read is "Masterminds of Programming" (though now that I think of it, this is probably due for an update with the amazing array of new languages modern developers can choose from!)

Masterminds of Programming

It's a great collection of interviews with different programming language designers, and (in addition to fun quotes of GvR trash-talking Java, and Java designers trash-talking C++ and Bjarne Stroustrup, etc.) it illuminates just how strongly the design of different languages are shaped by:

Personalities, preferences, heuristics
Original design purposes
Problems/objectives to solve
Constraints of the time

Javascript was created in a handful of days at Netscape, for example! It would be a very different language if it was designed by committee over the process of several years. But it's been very successful because it solves a specific problem with great integration, very minimal syntax, and good libraries for DOM manipulation and asynchronous programming.

Lower-Level Languages

Let's start with the "bottom" level. We'll assume that we don't necessarily need to learn assembly or machine language. Instead, what makes a language "low level" is typically something like this:

It builds to a platform-specific set of instructions
It is compiled (and possibly linked)
There is no garbage collection or similar "guards"; you are directly allocating and manipulating memory

C and C++ are obvious examples here. (The third bullet from above precludes a language like Go, but it does blur the lines a little bit.) Other good examples include Zig, Rust, or even Fortran. There are similar candidates but, in general, it's very useful to have working knowledge of at least one "low-level" programming language.

One caveat: You lose a lot of quality-of-life features at the lower level. One thing that C and C++ don't have is a modern package manager, for example! How are you identifying specific bundles of code and symbols? How are dependencies being resolved and combined? How do you conduct build and meta-operations (like documentation and testing)? Some languages have modern solutions for these, like Rust, but it isn't always easy.

General-Purpose Language

This is not a domain-specific language--quite the contrary. Languages living here are going to come with a degree of garbage collection and other resource/performance automation features. You will also have a variety of platform-agnostic tools for key OS interactions like path resolution, file input/output, threading, etc.

One key feature of a general-purpose language is just how quickly and easily you can address a specific arbitrary problem, particularly at the prototyping stage. This is often enabled and characterized by a healthy package ecosystem, which makes it easy to reuse your own code, and to leverage code that other people have written. Another distinguishing feature at this level is the availability of bindings to other languages--being able to wrap a static library from C, for example.

This is a great place to do side projects and other experiments. It's also a great place to develop comfort without needing to look up documentation for specific syntax and other subtle variations. Other examples (beyond Python) include Ruby and Go. You'll also find a lot of small teams built around these languages because they're commonly fluent and easy to recruit for.

UI Scripting

At our "highest" level, we have a set of more elementary languages that (while not necessarily domain-specific) are commonly used for binding things like UI events, glue between systems or applications, and other automation capabilities. There is probably some kind of embedded context or interpreter here, which is one of the defining characteristics (compared to general-purpose languages that are more likely to be running on an intermediate byte-code representation).

Other examples of higher-level UI scripting languages, beyond Javascript, include things like Lua or (arguably) Swift. I would even put React (let the debate begin!) in this category as its own unique quasi-language (mixed syntax aside, this isn't obvious). And of course TypeScript qualifies as well.

BUT WAIT THERE'S MORE!

There are a few other languages that will be useful for you to learn in your career. These don't really correspond as well to specific "levels" as the first three. But for context let's consider a model of game engine organization I put together in an earlier article.

https://dev.to/tythos/engines-evolution-10gk

In particular, let's consider the following diagram that defines an engine evolution that represents the boundary between beginner-intermediate game development efforts and intermediate-advanced.

There are several "layers" to game development. The "engine"--which is largely a way to scaffold event loops, window management, shared interfaces, and libraries of reusable software elements--isn't actually most of the game.

The engine is focused on performance and reusability, but it is not focused on implementing the mechanics of a specific game. The game logic, specific models, and other things that are developed to realize a specific game publication, are probably written "on top of" (or at least in conjunction with) the lower-level engine code.

But, there's another level beyond this. If you've ever written add-ons in World of Warcraft, for example, you know that there's an interpreter embedded within the game mechanics (with specific events you can subscribe to and assets you can extend or customize). Lua is a great example of this higher-level "scripting" usecase that can be used by the community to customize and extend different behaviors.

If you only know languages that quality for each of these three layers, you're going to do pretty well. But there are at least three other languages that will come in handy. This will be true regardless of what your "tech stack" selection of languages looks like from the first three "layers", whether it's C++/Python/Javascript, or Zig/Go/Lua, or anything else.

Shell Scripting

Purely for the sake of "getting around" your development systems, you will likely need to learn some degree of shell scripting. If you're on Linux, for example, learn to write a .sh script. On Windows, learn how to write a .bat file. Automating system actions (including basic variable assignment, pipe manipulation/integration, and control flow statements) is an invaluable way to save time and headache. Many, MANY tasks you will need to function will greatly benefit from the ability to automate it with a small shell script.

Interestingly, Python started out as a cross-platform scripting language for the scientific computing community! While it's since grown into a GPL (a lot of languages do this, much like Zig is growing into a GPL having started out as a systems-programming language), it still has very potent capacity as a scripting language for automation (think about the os, sys, threading, and subprocess core modules, for example). Bonus points for crossing off two categories with the same language, of course!

Domain-Specific Languages

In contrast to "general-purpose" programming languages, there are "domain-specific" languages that are uniquely suited (and designed around) the defining usecase of a specific domain. MATLAB is a good example here, which you will likely need to learn to perform heavy lifting in numerical algorithms and tensor manipulation.

Tensorflow is also an example of a DSL, even though it's used and manipulated in other languages, because there's a specific grammar and syntax to the manner in which neural networks are constructed, configured, and trained/utilized. You could learn Terraform as a DSL for large-scale static infrastructure-as-code (IAC) automation, for example, if you do a lot of cloud engineering. If you like crypto/blockchain, you'll likely pick up something like Solidity at some point.

But by definition your DSL will likely be a function of what specific domain you find yourself. Just be aware that you're probably going go benefit from picking up a DSL at some point.

Structured Query Language

The way you interact with (and understand) data, data-oriented development, and data-driven engineering is largely facilitated by learning SQL.

No other category in this article requires you to choose a specific language. There are many candidates for scripting languages, general-purpose languages, etc.

But you need to learn SQL. Because it rewires your brain in an incredibly helpful way.

You need to understand how queries work; how data is sorted and filtered; how tables and schemas are structured; how relational references work; how different "JOIN" logic interacts; how ACID assurances affect how data manipulation and mutation takes place.

And NoSQL and document stores are fine things to know as well. It's helpful to learn how OpenAPI and JSONSchemas work. These are all specific data structures and interfaces, more than specific programming languages. But regardless of whether you like to use PostreSQL or MySQL, you should learn SQL.

SQLite is a great place to start, by the way. Among other things, you can manipulate SQL directly within a memory store, or directly against a specific file, and it's API is built into common languages like Python. (This, by the way, is one of the ways you can tell how good a general-purpose language is--by what kind of tools it gives you "out of the box" to interact with common data and procedure interoperability requirements).

So. Choose any other language for any other category on this page. But learn SQL.

Conclusion

Let's review. There are three "layers" of languages that you should learn at least one candidate for:

"Low-level languages" that are typically platform-specific, compiled, and "direct"--that is, free of garbage collection or other resource abstractions. (Sometimes you'll hear the term "systems programming languages" used here as well, by the way, though that is more specific.)
"General-purpose languages" that are quick and easy to use, typically interpreted in byte code, and come with a healthy package management ecosystem for reusable tools. (Your operating system will not be written in a GPL. But the main mechanics of your favorite game might be.)
"UI/scripting languages" that help easily bind high-level UI interactions and other events with basic behaviors and interactions that "glue" together other languages and resources.

By the way, there are specific ways for each of these layers to "interoperate" with one another, as well. Whether it's REST and WebSocket interactions between the top two layers, or C ABI and structure-sharing between the lower two layers, the "health" of your "stack" will likely depend on how well you can combine specific tools in each of these categories.

There are also three categories of languages beyond these "layers" that can be very helpful to consider.

Some form of scripting language for automating programs and other system interactions for your operating system
Some form of domain-specific language, or DSL, that is specific to your particular field or problem set
SQL, in order to learn how to "think" in data-oriented operations and structures.

So that's about it. Learn them and love them! A good combination of these categories will take you all the way through your career. Unless you decide to dabble in the dark arts of functional programming, at least.

[14/52] Fundamentals of Continuum Engineering

Brian Kirkpatrick — Mon, 05 Aug 2024 23:34:00 +0000

Definitely something a bit different today! We're going to look at the game "Railway Empire", and I'll try to use it in explaining a key concept in continuum mechanics.

What are continuum mechanics, you wonder? I'm so glad you asked!

A continuum is an arbitrary volume (or boundary) of space characterized by a state space defined via continuous vector field. Continuum mechanics are the mathematical means by which we characterize real-world phenomenon by describing what happens within this space using that vector field. In particular, we want to anticipate how the vector field behaves over time.

This is a very generalized concept, of course! It comes down to two key concepts:

What is the volume of space you are concerned about?
What is the vector field? Or, in other words, what is the "stuff" you are interested in characterizing within this volume of space?

Depending on your answers to these questions, you can come up with insights into a wide variety of studies, from incompressible fluid mechanics to plasma fields to traffic & logistics problems. The underlying mathematical ideas are very powerful, but they can be tricky to wrap your head around.

Maths

Let's start with three interesting, but somewhat complicated, equations.

RTT is "Reynolds Transport Theorem". Basically, the change over time over the volume itself of a particular vector field can be expressed as the sum of two terms: The integral of the change of that field over the volume, and the integral of the change in the field over the flux through the boundary.

It looks crazy, but this is just an application of the product rule from calculus across the boundary as it changes. The big decisions here are, again, what is your vector field f and how does it relate to the changing boundary Omega?

CME is the Cauchy Momentum Equation, which captures conservation of momentum of a fluid within a particular volume boundary. Not that different, once you recognize the terms!~
NS(ic) is the incompressible, conservative form of the Navier-Stokes equation. And what do you know, there are similarities! The changing quantity (on the right) is some change over time plus the changes across the boundary as the field changes with respect to itself.

Rails

But these are just symbols. Let's take a very simple case here and go back to our game. We're looking at "Railway Empire", something of a successor to the classic tycoon games. This is a Germany map with specific geography and resources, but the mechanics of the game lend themselves to continuum characterizations.

There, in this screenshot, is a rural station generating cloth. That cloth is shipped to the nearby city Frankfurt, where it is used to manufacture clothing.

This is a particularly difficult scenario in which you need to get ten cities to a population of 120,000. There's a lot of emphasis placed on understanding and optimizing the flow of goods to achieve this, and it's not easy to do. There are many different goods (a vector space, in fact!) that need to be considered to satisfy the demands of a city to the point where it can grow to be that big.

In the case of cloth, the tailor in Frankfurt is also making goods that will be in demand by other cities, so trains may carry cars of that good to adjacent Wurms, Cassel, Cologne, etc. How do we characterize and balance all of these variables?

Logistics

Let's consider a boundary about the city. This defines a volume (if one in two dimensions). Let's focus on a single variable in our vector space at first, clothing. There is some rate at which clothing moves into this boundary (we'll use the variable c at first). Let's say c_in is the rate at which clothing is entering this boundary. There are also goods moving OUT of this boundary, which we will characterize by the rate c_out.

There will also be some rate of resources as generated by the city's factories--we'll call this c_gen. And there will be some rate of resources consumed by the city's residents to satisfy demand--we'll call this c_cons.

Because these resources must be conserved, the overall rate of change (which we want to be either positive or constant) can be described as:

c = c_in - c_out + c_gen - c_cons

We can look up specific details that let us define values for each of these numbers. If 1.2 units of cloth are needed for manufacturing, and 2.1 units are needed for consumption by the city, then (if we assume no cloth leaves the city directly) we know c_in must be about 3.3 units/week.

Fun With Units

We can also look up other variables that let us "solve" for the transportation required to satisfy this 3.3 units/week requirement.

In this case, we see (from the four trains delivering cloth from the source to the city) that about 100 days (or about 14 weeks) are required for a round trip per train. With 8 units (or cars) per train, we compute the following "flux" into the city:

4 [trains] * 8 [units]/[train] / 14 [weeks] = 2.3 [units]/[week]

Starting with a known constant, we can compute the rate and use the units in conversions effectively as variables themselves, cancelling them out to reach the appropriate measurement. More importantly, we can replace 4 [trains] with n [trains] because we may want to solve for how many trains are required to satisfy the demand for that city--that is, to increase the flux of this metric across that boundary.

Other Constraints

There is only so much flux the rail network can handle before a "shock" develops. You can start exceeding the capacity of the rail line, with implications for (if time at junctions becomes an issue) how trains can back up into each other. Number of tracks at the station also becomes a limiting factor (or "LIMFAC" as we call it in engineering).

In that case, you need to add other means to enter and exit the city. In this game, that means you need to add warehouses and similar structures. This effectively "expands" the boundary we were considering in our integrals! You can repeat this analysis by considering the resource exchange at the rural stations, where resources initialize, or by considering multiple resources (e.g., having a vector field with more than one dimension).

In Summary

To summarize Reynolds Transport Theorem: To consider the change of a vector field across a changing boundary, you add the change of the field over time with the change of the boundary over the field. This is just a conservation law, though, that we can also capture with the continuity equation:

We expressed these as discrete signed rates in our equation for c, but you'll notice:

c_in - c_out

...doesn't need to be signed in our vector space calculations because we are integrating the flux across the boundary--they're the same thing captured within the integral. Similarly:

c_gen - c_cons

...is simply the change in the integrated quantity over time. (This is particularly relevant in thermal problems where heat is generated and consumed by different reactions that may be taking place.)

You can also imagine how these concepts might reappear--not only in our upcoming "units" segment, but as we address problems in fluids, logistics, and other fields.

Simplified

We have four key concepts we can define to simplify all of the above:

There exists some CONTROL VOLUME, which we can think about as the area around a specific city or station
There exists some PROPERTY, like an in-game resource (cloth or clothing are two examples) or (in a fluid) momentum, mass, or other conserved quantities
There exists some INFLOW and OUTFLOW of this property across that volume. Trains are bringing resource into and out from the city. And moving fluids may be bringing momentum, mass, heat, or other quantities across the changing boundary of our volume.

Finally, REYNOLDS TRANSPORT THEOREM helps us describe (compute, predict) the behavior of this system by tracking how the amount of resources change over time based on what's coming in and going out.

Combine these concepts, and you can manage your resources better within challenging Railway Empire scenarios and make strategic decisions about how many trains you need, how many stations define the boundary of a city's volume, and how cities can efficiently and profitably grow within the game. Cool stuff!

[13/52] Fundamentals of Systems Engineering Models

Brian Kirkpatrick — Sat, 03 Aug 2024 00:19:41 +0000

We're going to continue focusing on key overall concepts in engineering. We're targeting a 2nd- or 3rd-semester engineer undergrad survey level of engineering content, so you may find just a touch of calculus helpful, but it's not required. Instead, I want to try and introduce as many interesting engineering concepts to readers and viewers across the field!

Today we will focus on four particular models from systems engineering that have incredibly useful applications to your thought process and the way you see the world. These are specific ideas that can be considered in both quantitative and qualitative ways. I've selected these four models based off of how useful I've found them across a wide variety of interesting problems and situations, both in engineering and in life.

State Space Representation

In a state space representation, we are considering how a controls process can interact with the system it seeks to control. The first element of the state space representation model represents the "PLANT" or "universe"--basically, everything external to our controller.

Within the universe of things outside our controller, there is some specific subset of states we want to control (more on that later). Specifically, there is some desired state and we want to take action to ensure this plant (like a robot arm, or factory furnace, or anything else) is doing what we want it do--that is, it is operating towards the desired state.

There is some subset of information that you (as the controller) can "perceive" about this plant. There is some subset of measurements that you can determine using specific systems called "SENSORS". Those sensors are systems of their own, with their own transforms and behaviors, and from those sensors you draw specific information about what you want to control. These are effectively your inputs into your controller.

Next up is the "CONTROLLER". The information from the sensors goes into the controller, whether it is digital or analog (or anything else). The controller is responsible for making decisions about how the "desired state" is to be achieved, based on the input received from the sensors.

As an aside, sometimes the "state" is characterized by a variable x (often it is a vector of numbers). The controller is rarely able to perceive the state directly; rather, there is an output (y) from the sensors. And based on the controller's understanding of the plant from the output y (which is often "inverted" in some way to divine the underlying state x), combined with some "desired state" x0 (or specifically the difference between the estimated state and the desired state, dx), the controller will issue some sort of command to another set of systems.

Those systems are called "ACTUATORS". Much like an actuator in a robot arm, these are systems that take some command signal to perform an action that influences, in some small but controlled (and well-characterized) manner, the state of (or input into) the universe (and/or plant).

Often you'll see this "input" labeled u. One "form" in controls engineering you'll often see is a representation of system dynamics using the following equations:

dx/dt = A * x + B * u

y = C * x + D * u

We are making some assumptions here--for example, we assume the system is linear and time-invariant so we can represent A, B, C, and D as constant-value matrices. These assumptions will be different in other kinds of systems, like non-linear systems (in which case, A might be a function of x). There may also be other input signals--for example, forcing functions or noise from the world or other environmental factors. Regardless, just in this form we have a very powerful target for mathematical tools to solve for ideal solutions to our controller.

Another key observation is that the dynamics of the system (x as it changes over time, characterized by the first equation being a differential) are rarely the same as the input or controlled signal (u), and are rarely the same as the signal we can observe (y). But this model captures all of these phenomenon quite neatly.

There are also different dimensions to each of these signals! Your control authority may only affect a subset of the desired state. This leads to interesting implications about what is (and isn't) controllable. This also captures the solution to Kant's fallacies in his critiques of reason--he asserted that we cannot truly know because we do not directly perceive, and are therefore limited to the information (for example) our eyes provide to us. But, if we understand our eyes well enough, we can correct for the "sensor" behavior and determine a much more accurate understanding of the "universe" truth or state.

Optimal Control Model

I had a song I sang to my kids when they were very young kids--"when you want to go from point A to point B, you take a taxiiii!"

When you want to go from point A, and find yourself at point B some time in the future, that transition takes place through a space (perhaps a state space). That transition is a path through that space. Effectively, this is a boundary value problem and we are solving for a path.

That path will have some sort of cost associated with it--usually a function of each point on the path that can then be integrated to determine the cost of the solution. That cost (whether its dollars or some other non-monetary value) can then be subject to a minimization process, using many interesting and powerful techniques in calculus. (Of course, this will only work for a scalar cost! Reducing multi-objective optimization problems to a scalar cost is a tremendous challenge in underconstrained problems like design.)

So, we have three challenges, regardless of the space or how quantitative we are:

Where are you? (What is point A?)
Where do you want to go? (What is point B?)
How will you get there? (What is the path? What will it cost? And how can the path be chosen to minimize that cost?)

This is even relevant to issues as qualitative issues like policy. Everyone always wants to talk about B (where they want to go). Utopia! But no one wants to figure out where they are (A) or how they got there. And very, very few people have the expertise or competence to characterize, predict, or solve for a system that successfully manages a transition from A to B with an acceptable cost.

Standard Stream Model

The next process model comes from computer science. Let's say you wanted to write a basic computer program in C++, something along the lines of the classic "Hello, World!":

#include <iostream>

int main(int nArgs, char** vArgs) {
    std::cout << "Farewell, cruel world!" << std::end;
    return 0;
}

Then we could build and run it like so:

$ gcc main.cpp
$ .\a.out
Farewell, cruel world!

As simple as it is, this captures a number of interesting facts about the way a computer program, as a transform, is modeled:

There is no input here but we could theoretically pipe in inputs from some other program or file contents
We do have outputs that are being printed to what is called the "standard output stream".
We aren't processing them but you can see the "args" values that would let us consider different flags or options
We also have some 'dependencies" that we include (or import) to define how our own particular transform takes place

So, let's characterize this transform model There is some input, which we will label STDIN in homage to the standard input stream. This results in the generation of output, which we will label STDOUT for similar reasons.

But depending on what you want the program to do, there may be other options that characterize (for example) the output format or other settings. These represent the "configuration" of the process (like the knobs on a machine) that customize HOW the transform takes place.

We will also have dependencies that the transform leverages. These effectively are a decomposition of the top-level transform into a variety of other transforms.

And lastly, you may have noticed we didn't mention STDERR. Often a transform may have some other form of "meta-output" or log messages--a different pathway by which a transform generates information that isn't directly part of the STDOUT. This could be controlled by syslog levels, for example.

This process model is incredibly useful and applicable to many different kinds of transforms. For example, this can be used to characterize a specific station on a factory floor. Or perhaps this is a stage of automation in a digital engineering activity. Any process that transforms information can use this model to characterize what it is doing. (Not to mention any transform that is captured by software!)

Systems Engineering V-Model

Our fourth model is a "v-model" that characterizes creative activities. This is specifically derived from systems engineering but is universally applicable to any human creative endeavor.

First, you have some form of design. External inputs (like customer desires or architect drawings) characterize some sort of design that goes into an engineering process.

Second, that engineering process results in the development of a solution. This could be the formation of specific schematics, a manufactured product, a webpage, or some other software application. This is a creative process because the inputs are typically under-constraining the problem at hand, and therefore engineers require some combination of heuristics, experience, best-practice, and judgement, and iteration to achieve.

Finally, the internal development process hands of the result to a delivery to or with external stakeholders. Because this goes "down and back up", it is typically depicted in a "v" shape, and while we only described a simple three-step model here, there are many, MANY variations on this model that introduce considerably more complexity.

What are some examples, you ask? Why, I think we can show you!

Welcome to the nightmare of the JCIDS. This characterizes the process by which the government engineers and produces products for defense. It's very complicated--but nearly every step is characterized by a different variation on the "v" process. It's relevant to every step of the process--from research, to requirements development, to development, to initial production, to full-rate production, to sustainment and retirement...

This model is also relevant to software. If you look at any sort of issue model (whether it's on Github, or Gitlab, or scrum, or any other sort of process), the "issue " lifecycle will look like this "v". Let's consider the classic "Bugzilla" bug lifecycle model, for example:

Stages 1-3 are "external"--a report or issue is generated (or designed), or maybe a new feature is proposed. In steps 3-5, the solution is developed by an engineer assigned to the task. Finally, in steps 5-7, some external verification and assessment (or acceptance) activities take place.

Even if you don't care about engineering--say you are a sculpture. Your client asks for a statue that will go in their garden. They'll tell you how big it needs to be, what aesthetic it needs to suit, etc. That high-level design is then considered by you as you start iterating your block of marble. And hopefully, you'll deliver that statue to the client resulting in great satisfaction for everyone.

Summary

Let's review our four important models:

The state space representation (or SSR) shows how a controller and related systems are put together to achieve some desired state of a controlled system, or "plant", through a combination of sensors, actuators, and controllers that operate on (or at least consider) some combination of input, state, and output signals.
The optimal control model defines how a system can transition through state space from point A to point B, using a path with an associated cost C that can be integrated and minimized. (An "optimal" control is one that achieves the globally-minimal cost, by the way! Bonus points.)
The standard stream model (or SSM) uses computer science to characterize a transform from input to output, as defined by specific configurations and dependencies, along with an optional "meta-output" or error stream for logs and monitoring.
The *systems engineering v-model" (or SEVM) shows the process by which external requirements and objectives, initially defining an under-constrained design problem, are subject to development in order for engineering to generate a solution that is delivered back to external stakeholders.

These four models are incredibly useful, both in systems engineering and just in general thinking about life. They are also somewhat related--for example, when I talk about concepts like "state space" that are relevant to multiple processes. And once you "grok" these, you'll start noticing them everywhere in life! Many people know about one or two of these, but being able to put them all together turns your brain into an engineering superpower.

[12/52] Fundamentals of Knowledge Engineering: Introductory Graphs

Brian Kirkpatrick — Wed, 24 Jul 2024 23:27:52 +0000

Today represents the first in a long-promised transition of the series away from purely software development problems towards more generalist engineering topics. And I'm excited to finally get going! It's already most of the way through July, so I have a lot of catching up to do if I want to hit 52 "episodes" by the end of the year.

Latin Quizzer

When I was a senior in high school, I took Latin 1 & 2 to see if it would help me boost the verbal section of my SAT score. Most of my classmates were starting Latin in 6th grade, though, so I was way behind and needed a way to catch up. I wound up coding my own flashcard-like quiz app in GW-BASIC (I was just staring to learn C/C++ at the time). Even though it was just a quick "this term translates into what other term?" string evaluation--long before Duolingo was a thing!--it was incredibly useful.

Fundamentals of Engineering

I'd like to take a similar approach as we launch on our journey through the fundamentals of engineering. Let's think about writing a "quiz generator" application logic of some kind. How do we need to break down a topic area (whether it's electronics, or structures, or anything else)? We need some representation that lets us "generate" problems. There are a couple of useful ways to think about this.

Problem Generation

We'll consider a few different "types" of problems. Each problem will have a combination of "concepts" and test the student's knowledge of these concepts and their "relationships". The easiest way to think about this is with a specific equation:

P V = n R T

In this example, we have five "concepts" (pressure, volume, count, gas constant, and temperature). These concepts are related to one another with a specific "relationship". In this case, this relationship constrains those concepts (much like we saw in our earlier "Spicy Math" video). Five variables (or "degrees of freedom") in a problem mean we would automatically generate values for four of them, then ask the student to use knowledge of this relationship to find the unique solution for the final degree of freedom.

Knowledge Graphs

When we talk about concepts and relationships, you've likely formed a picture of a graph in your head. Keep in mind we will have different "types" of relationships we want to explore (not just equations). Nonetheless, this idea of a "knowledge graph" is a powerful way to model content within a specific topic area.

But we can't "connect" specific nodes to other nodes because those connections (or relationships) themselves are uniquely modeled areas of our knowledge. Instead, we have multiple "sets" of nodes--at least two, even if we just focus on equations. This type of graph is commonly called a "bipartite graph", in which connections exist between sets of nodes (but not within the sets themselves).

Measurements as Nodes

One "set" of nodes in our graph will be the "concepts" themselves. Let's focus on our ideal gas equation. The "concepts" in that case were specific mostly measurements, like temperature and volume. (The exception of course was the ideal gas constant.) So, this defines the first "set" of nodes in our graph. We might model some of these measurement like so:

{
  "id": "P",
  "name": "pressure",
  "unit": "Pa",
  "type": "variable"
}

In the case of the ideal gas constant, we might want a slightly different model:

{
  "id": "R",
  "name": "Gas constant",
  "unit": "J/(mol*K)",
  "type": "constant",
  "value": 8.314
}

For easier lookup and parsing, let's assume these models are captured in a CSV table our app will consume. Empty columns will be unused for nodes that don't have relevant properties (pressure, for example, has no constant value).

There may be other "concepts" we want to model as nodes. For example, we may want to assert that a specific measurement "has a" specific type of unit. Or, a unit may "belong to" a specific set, like the set of all base SI units. In each relationship, both "ends" will be concepts, and their relationships will be specific kinds of connections linking those concepts.

Relationships as Equations

The next "set" of nodes we will want to consider are the relationships themselves. In our ideal gas equation example, the equation is a relationship that constrains (connects) specific concepts (or variables and constants). How would we model a relationship like this? Maybe like this:

{
  "id": "ideal_gas_law",
  "type": "equation",
  "equation": "PV=nRT",
  "involves": "P,V,n,R,T"
}

We are ignoring, for the moment, the semantic issue of evaluating these equations symbolically. Instead, we'll assume the relationship can be "evaluated" at runtime to determine if the user's answer was correct. (This is one reason we include a specific expansion of the variables this equation involves.) Some degree of variation or symbolic manipulation could be introduced, though, in order to mix up which specific values must be "solved for".

Memberships

Just like there are other concepts we may want to model, there are other kinds of relationships we want to consider. For example, we may want to assert the membership of R in the set of all physical constants. We may also want to specify that the concept of "milliliters" is related to the concept of its corresponding "base SI" unit, liters, by a specific conversion factor.

Memberships are a key relationship but expand the "set" (beyond just two) of relationships we model between concepts. Let's assume for a moment this is only one degree of abstraction.

We can also recursively expand upon memberships, though, and all other relationships for that matter. In fact, the "difficulty" of a problem could be determined by how may recursive expansions (additional relationships) the student must consider when determining the correct solution. Maybe pressure and volume are not defined, for example, but must be derived from some other relationship (like an adiabatic expansion).

Problem Sets

Let's think about specific problems we might want to generate. Depending on the problem, different relationships will be draw and presented in different ways.

Solve an Equation

We've mostly focused on a "solve this equation" problem so far. This would "draw" (at least one) equation from our table of relationships. It would then randomly generate values for all but one variable. (This is also implicitly true for "units" type problems, even though the relationship in that case is not an equation but a factor conversion--unless you're talking about Farenheit and Celsius conversions, of course!) The "givens" would be presented to the user, who would be asked to determine the "missing" value. This value would then be plugged into an evaluation (literally an eval() call within Javascript, perhaps) of both sides from the equation relationship. These sides would then be compared to each other to determine, within some tolerance, whether the user was correct.

Determine Set Membership

We could also use set memberships to generate "multiple choice"-style questions. Given a particular assertion, several values would be presented, some subset of which would be true for the given assertion. For example:

The following are base SI units:
- [ ] Milliliters
- [ ] Pascal
- [ ] Yards
- [ ] Metric Tons

Given a set relationship, we would draw upon multiple "concepts" (some of which would be directly related by membership, others of which would not be). If only one is directly related, we would have a "check one"-style question instead, which is considerably easier.

Statement is True?

The last kind of question type would be a combination of both. "Given" a certain set of values, an assertion is made, and the user must determine whether or not that assertion is true. For example, "will this projectile land within X yards of a target Y yards away given certain kinematic parameters?" could be considered. In this case, the algorithm generates a problem for which it knows the answer, but presents a value (which may or may not be correct), including true/false assertions, that the user must consider.

Units Examples

Let's consider the "Units" topic area. Let us assume we have "drawn" a relationship from our table. In this case, it is a "hierarchy" relationship that expresses a conversion from a unit type to the corresponding base SI unit.

// 'draw' relationship
let relationship = {
    "id": "milliliters_to_liters",
    "type": "hierarchy",
    "to": "liters",
    "from": "milliliters",
    "factor": 1e-3,
    "prefix": "milli",
};

Given the "from" and "to" references to specific topics, we would then query the specifics of those "nodes".

let to = {
    "id": "L",
    "name": "liters",
    "unit": "L",
    "type": "unit"
};
let from = {
    "id": "mL",
    "name": "milliliters",
    "unit": "mL",
    "type": "unit"
};

We seek to generate a problem statement, but must first determine the values for the other degrees of freedom. Let's assume this is somewhere between 0.1 and 10.0, which is a reasonable range of values for most measurements. (We ignore negative values here, but there are concepts--like speed--that we could model to consider negative values.)

// given measurement x in units X, convert to measurement y given units Y
let x = Math.pow(10, Math.random() * 2 - 1); // random draw between 0.1, 10.0
let X = from.id;
let Y = to.id;

Now we could construct the problem statement and the appropriate solution. This would likely live in a more complicated set of switch-case statements that would consider what kind of relationship is being used to generate the problem.

// construct problem & solution
let problem_statement = `Given measurement ${x} [${X}], what is the equivalent measurement in [${Y}]?`;
let solution_value = x * relationship.factor;

Lastly, we would "present" the problem statement and evaluate a result "given by the user" (use your imagination!). Note in this case we specifically look at a relative error tolerance to determine if the user's answer was correct.

// 'render' problem
console.log(problem_statement);

// 'evaluate' result
let result = 0.1; // pretend this came from user
let error = Math.abs(solution_value - result) / solution_value;
if (error < 1e-2) {
    console.log("RIGHT!");
} else {
    console.log(`WRONG! The correct answer is ${solution_value}.`);
}

Conclusion

I won't re-generate this kind of video for each topic area of engineering we explore. I will, however, try and keep the "database" of knowledge graphs somewhat up to limited date with the content we cover. At the end of this series, this will hopefully provide you with some degree of training and verification of the engineering knowledge you are consuming!