starting overview of my new project animal_game

dexter — Thu, 21 Sep 2023 13:08:34 +0000

The goal is to create an ecosystem simulator similar to Rainworld. Maybe turned into a game if I get an idea that wouldn't just copy Rainworld. The main purpose is to learn (however with the ambition to reach a working product). One of the side goals is to find a better name.

This project currently consists of three repos. The game/simulator. The minimalistic game engine written in parallel for this project. The macros repo for ECS abstractions.

I'll try to write blog posts about this project to...

practice writing in English
get additional motivation from these little milestones (blog posts about what I did or maybe what my plans are if they're interesting enough for a post)
to be forced to think through ideas well enough to be able to explain them
the tutorial style posts maybe will help someone (like this one that's actually the first post about this software project) and the rest might be interesting to read

some technical aspects

This project will use

Rust, because I want to further learn the language, my experience is limited to reading the book and an unfinished tetris game
the Entity Component System pattern, I'm intrigued by the concept and want to try it
a minimalistic game engine written in parallel that's build on top of ggez, I want to learn how to implement the physics part of a game and I already used ggez. ggez is just there for the fundamental parts like game loop or drawing graphics, the rest is something I want to learn myself.
decentralized algorithms for animal behaviour, I'm currently very interested in decentralized, self-correcting, etc. systems/algorithms/... and want to learn about it (this was the main reason why this project idea won against others)
clean code and architecture, I've read about these concepts in multiple books by Robert C. Martin, but until now only used it to rewrite my last project in the middle of development which turned it from pure hell to something nice, this time I want to create a clean system from the start

In terms of software architecture (especially interfaces), my current approach is to look at what I need right now (and maybe in the not so distant future) and built it in a way that enables future extensions and easy switching of components. With that I hope I don't get into perfectionistic planning hell while ensuring a clean architecture. Let's see how this approach will develop and how/whether it will work.

current plans for the game engine

graphics:
- draw various objects including abstract ones (light, fluids)
- varying window sizes
physics:
- applying forces: hitting, throwing, grabbing
- dynamic vs. static objects
- fluids
- friction (no idea if friction is doable or how to do it, in school this thing did not exist, I have no idea about it)
animal bodies:
- bone system for movements
- body rendering that uses individual body parts instead of whole sprites

current plans and ideas for the game/simulation

The game itself is about visualizing animals and environment and (the main part) animal behaviour.
The current ideas include (not exhaustive, a lot taken from Rainworld):

basic senses: seeing, hearing, touching?, smelling?
basic reflexes: attack vs. run, hunting/eating
more complex behaviour
- personality: small set of fundamental characteristic aspects that influence the behaviour. E.g. impulsiveness, willingness to take risks
- social memory: remembering interactions with others to cause things like rivalry, groups, etc. (maybe even revenge)
- dynamic food chain position: no fixed positions, a worm should in theory be able to kill a giant somehow
somehow evolution (not sure if some parts are doable)
- implement mating and giving birth functionality (the foundation for evolution)
- newborns get a mix of their parents' personality with some randomness
- try to make physical appearance affected by evolution
- try to make dynamically developed instincts

How to abstract away library interfaces working directly through syntax in Rust with procedural macros

dexter — Thu, 21 Sep 2023 11:41:32 +0000

In this post I describe how I wrote two macros to separate the bevy_ecs interface from my app core even though bevy_ecs' system interface works directly via the function syntax.
This isn't a complete tutorial for Rust's syn and quote libraries, the Entity Component System pattern or Rust macros. I won't explain every struct that I use. Read the library documentations for that.

You can find my source code here.

general info about procedural macros

In both cases I'm using procedural macros to remain flexible. With bevy_ecs a derive Component macro would probably suffice, but some ECS libraries presumably use another way which would cause problems if I tried to switch to them.

Procedural macros get two token streams (one for the macro arguments like Debug in #[derive(Debug)], one for the block that has the macro) and return one (the code that replaces the code block which is affected by the macro). They look like this:

#[proc_macro_attribute]
fn macro_name (args: TokenStream, input: TokenStream) -> TokenStream {
    //...
}

I haven't used macro arguments yet, so I won't describe them.

To parse the incoming token streams into something usable like a token tree, Rust offers the syn package. For the opposite direction Rust offers the quote package.

To get from the token stream to a token tree, syn has multiple parsing macros. I used parse_macro_input!. The main difference to the others is that parse_macro_input! enforces what type of syntax block you get. E.g. parse_quote! just parses any string.
A call would look like this:

let parsed_input = parse_macro_input!(input as ItemStruct);

In this case my custom macro is expecting a struct. I parse the input parameter to ItemStruct and parse_macro_input! checks if input actually is a struct and returns an ItemStruct instance. ItemStruct is a normal Rust struct provided by syn that can be nicely worked with!

After you've edited your token tree or created a new one you have to create a new token stream that will be the output of the custom macro. There are multiple ways for different scenarios.
The most simple one is using the quote! macro.

let parsed_input = something_that_implements_ToTokens;
let input_struct_identifier = again_something_that_implements_ToTokens;

let new_tokenstream = quote! {
    #parsed_input

    impl #input_struct_identifier {
        fn im_just_here_to_show_what_quote_can_do (a: i32) -> bool {
            a < 100_00
        }
    }
};

In quote! you can directly write Rust syntax and use variables that implement the ToTokens trait (parsed_input and input_struct_identifier in this example). quote! substitutes these variables with their token streams.
The example above could produce a token stream resembling something like this:

struct SomeStruct {
    //...
}

impl SomeStruct {
    fn im_just_here_to_show_what_quote_can_do (a: i32) -> bool {
        a < 100_00
    }
}

However, if you created a struct that already implements the ToToken trait you can directly parse it into a token stream.
E.g. like that:

item_struct.into_token_stream()

component macro

The component macro is rather simple. bevy_ecs uses the same style that I want, so the macro simply adds a #[derive(bevy_ecs::component::Component)].

let expanded = quote! {
    #[derive(bevy_ecs::component::Component)]
    #input
};

There is probably a way to add the derive macro in the ItemStruct instance, but I didn't try that.

system macro

The system macro is far more complex and interesting.
In general, I want my system functions like this:

#[system]
fn some_system (a: &ComponentA, b: &mut ComponentB, entity: Entity) {
    //a,b and entity are used here
}

bevy_ecs on the other hand, uses this style:

fn some_system (query: Query<&ComponentA, &mut ComponentB, Entity>) {
    for (a, mut b, entity) in &mut query {
        //a, b and entity are used here
    }
}

This has quite some syntax overhead that I don't like. Thus, I combined keeping dependencies away from my program with getting an interface more of my liking.

This comparison already shows how the token stream input looks like and how the output is supposed to be.

The macro can be seperated in creating a new function signature and creating the for loop. Then you create a new function that copies all the other properties from the old function and insert the new signature and new function body.
It could look like this:

pub fn create_fn_item (input: ItemFn) -> Result<ItemFn, TokenStream> {
    let new_sig = create_new_sig(&input.sig)?;
    let for_loop_stmts = create_new_loop_stmts(&input)?;

    let new_fn = ItemFn {
        attrs: input.attrs, // attributes like #[repr(transparent)]
        vis: input.vis, // visibility
        sig: new_sig, // signature
        block: Box::new(syn::Block{ // function body
            brace_token: Default::default(),
            stmts: for_loop_stmts
        }),
    };

    return Ok(new_fn);
}

I won't explain the steps for the signature and the for loop in detail because it would mainly be copying source code and writing some redundant comment. My biggest advise is to parse the structures you want and print them to the console. Then look how they are build up and combined with the syn documentation build your wanted syntax structures. You can use my source code as an example.

debugging procedural macros

Currently procedural macros can only be executed in their default way with #macro_name, even though they're just special functions. This means quite some problems for debugging because macros are executed at compile time. There are some debugging options but not a usual IDE debugger.
However, there is a workaround for that. The library proc_macro2. This is a wrapper around the procedural macro API, and it can be used outside a procedural macro. That means you can outsource your macro logic into plain functions and call these from your macro. The actual macro only does the initial syntax parsing, the conversion between proc_macro and proc_macro2 and calls the outsourced function. Because the latter is a plain and normal Rust function, you can unit test it or do whatever else you want.
This is an example how it might look:

#[proc_macro_attribute]
pub fn to_bevy_component(
_args: TokenStream,
input: TokenStream,
) -> TokenStream {
    let input = parse_macro_input!(input as ItemStruct); 

    let result = bevy_component::to_bevy_component(input);

    proc_macro::TokenStream::from(result)
}

pub fn to_bevy_component (input: ItemStruct) -> proc_macro2::TokenStream {
    let expanded = quote! {
        #[derive(bevy_ecs::component::Component)]
        #input
    };

    return expanded;
}

#[cfg(test)]
mod tests {
    //...
}

throwing errors in procedural macros

In case I mess up and use a macro on some invalid syntax thing I wanted error management via Result like usual. This doesn't mean syntactical problems but e.g. some syntax thing that isn't supposed to appear in the syntax things affected by the macro and that I therefore don't want to have to handle.
You can do it quite easily. You just have to create a syn::Error, call .into_compile_error() on it, parse it into a token stream and return it from your macro. For Rust to be able to determine the right spot that caused the error, you have to get the correct span. A span is some part of your parsed code that starts and ends somewhere. Usually you can get it from the syn structs.

I used this technique to exclude self parameters, because systems will never be associated to a struct (or at least don't offer any additional value than normal functions).

 match fn_arg {
        FnArg::Receiver(rec) => Err(
            syn::Error::new(rec.self_token.span, "systems can't have a self parameter")
                .into_compile_error()
        ),
    //...

Forem: dexter