I've got a dream!

My Dream

Once upon a time, I had a vision of 2 very different players, playing very different games...

YET! They were able to share items, laughing and enjoying interaction in their very different experience.

Prologue

To showcase my Grand Dream™️, I've got to build a demo!

I could make a simple local demo, sharing items offline, stored on the disk.

I could.

No.

I go big.

Multiplayer

So first things first, I install Rust.

Then I make a server.

Demo

So first things first, I make a small game.

Then I make a client library to connect to my server.

Then I plug things together.

Surprise!

So my plan was:

• Input a mail
• Input a password
• Create an account or connect to it
• Enjoy the API and relax.

Ah well, I have to tell you about my keyboard, I have an AZERTY one. To input @, I have to use Alt-gr.

I use bevy_egui for testing purposes (and there is no input on bevy_ui heh!)

But when I try to input @ with my keyboard, nothing appears!

That's a problem, because every email address have an @...

What's the root problem?

To keep it short, the problem is that winit handles Alt-gr by sending ctrl + alt, bevy_egui was ignoring all keys if ctrl was held. It makes sense, since ctrl is used as a "command" key on windows.

🎉 So I fixed that, First PR in this story! 🎉

Again.

I want to support web, but I realized my fix wasn't working on web...

I tried to fix it again on bevy_egui, but the handling of meta key in OSX was proving difficult to provide an exact solution.

Adapt, overcome!

OK, I spent quite some time on that issue now, I'm taking a break, keep a note somewhere and call it The Altgr Problem.

The copy-paste problem

By the way, my authentication process involves a password sent to the user so he can copy it in the password field.

On wasm, bevy_egui doesn't support copying from external source, the copy is only stored from and to local (process) memory. That's not useful to me!

What's the root problem?

I tracked down the problem to the ar board crate, which is not interested in supporting web because their API is fully synchronous, fair enough, let's fix it on bevy_egui's side!¹

Down the rabbit hole

Using web-sys crate and a few documentation, I had most of the implementation ready.

Most.

On web MacOS, bevy_egui currently handles copy-paste with ctrl-[XCV] ; it's wrong, because it should be Meta-[XCV], But at least it's working.

Now, reacting to clipboard web events fixes the inconsistency, GREAT right? I wish!

I discovered yet another problem within winit: Meta key not correcty forwarded from winit.

Winit

Thankfully, winit's folks have already detected all the shortcomings I found throughout this blog post, and a big keyboard input refactoring was almost ready.

At this point, I have 3 issues related to winit, most likely fixed with winit 0.29, let's try to update!

I asked around on bevy discord if anyone was working on it before rolling up my sleeves!

"Not a fun update"

François Mockers

I've been warned.

The tip of the iceberg.

The first task to upgrade our winit' API usage was to rename most of the Key enums. As bevy provides a abstraction over winit for modularity's sake, that meant a few regex search and replace.

With good tooling the task wasn't too tedious.

Thanks for the help

Thanks to bevy maintainers and other contributors, I could ask my way through:

• Copy trait issues
• foreign types to implement bevy_reflect on top of (SmolStr 🎉)
• getting a confidence boost when I was in doubt ❤️
• minor fmt fixes

I have to tell you about...

I could start a farm with all those rabbit holes I followed...

instant crate

winit replaced instant by web_time, I checked on the dependencies also using instant, updated what I could, and made a note on what I could not. cargo tree -i again my best ally.

AccessKit

Of course AccessKit doesn't support winit 0.29, it's not released yet (when I started).

So I started a PR, just enough to unblock my compilation, but sharing my work so it's helpful to others.

When winit released, the interest seemed high, and maintainers stepped in 🎉.

What if we could choose not to upgrade all ecosystem?

What if ecosystem crates could support every older versions of more foundation-y crates?

We could upgrade to the latest ecosystem, but still have a few outdated ones!

Right?

Well dependencies are hard, we could leverage rustc conditional compilation through cargo features to support all edge cases...

I see you shouting "UNMAINTAINABLE!".

Well.

winit folks did go the "hard way": they have features for raw-window-handle version 0.4, 0.5, 0.6 (a.k.a rwh_04, rwh_05, rwh_06).

it's cool, because I don't have to wait on wgpu to update to latest rwh_06!

So I chose to rely on rwh_05.

That said, I ran a quick cargo tree -i to see if my dependency tree war coherent... and rwh_06 was still pulled in.

winit and ndk had their default on rwh_06. I had disabled default features for those, but android-activity did not, so we worked on a fix 🎉.

"AAAAAA"

Everything is compiling at this point! What were we trying to achieve ?

I forgot, something related to keyboard input... Let's try the keyboard example.

It's working!

let's try to break it:

• Press shift
• press 'A'
• release shift
• release 'a'

Congratulations, I broke bevy_input. bevy_input thinks 'A' is still pressed: "Of course, you released 'a', a totally different letter!", the program told me.

My PR is already quite big, and fixing that might involve a few experimentation and controversial changes, So I chose to break it out, check it out!.

I went for my guts, and implemented a "dynamic cache", relying on the underlying keycode, but keeping in memory the current logical key.

Better API ?

Within bevy_input, we expose 2 api to interrogate which input is on which state:

• Input<Key>
• Input<KeyCode>

I took the opportunity to change it to a more unified

• Input<KeyLogic> where KeyLogic is either a keycode (or scan code) or a logicalKey (key displayed on user's keyboard).

This approach was quite controversial as it adds up maintainance burden, and most likely performance implication.

I raised my concerns on the Pull Request and on discord. After a few discussions, we dropped altogether adressing the "AAAAAA" problem, by mapping KeyCode to the PhysicalKey (place on your keyboard) rather than the LogicalKey (visible key on your keyboard), that's been actually wanted for a long time.

Hopefully when time comes, we'll find elegant solution to display to the user the visible key to hit in order to trigger an action. It's possible to retrieve a Physical/Logical key mapping on most platforms, but might be a security risk on web due to fingerprinting, it's not an "easy" topic!

A sizeable problem

That's a pun on resizable windows, yeah.

bevy (<= 0.12) had difficulties redrawing smoothly when resizing, I made the problem worse! I couldn't reproduce on winit, so it might be fixable on bevy's side ? Come help ♥

Thanks to early testers, we noticed I also broke web, a continuous back and forth concerning window logical and physical size was going on between wgpu and winit; thankfully I got help from winit's insider!

Window out, window in

Winit now supports launching a window again through run_ondemand, bevy still has to accomodate these changes.

Be careful with your scope when making a contribution! My objective was to update to latext winit. I took the Shortest path, new features will come next if possible.

Licensing

Hey, open source is hard work! Licensing exists and protects everybody from users to maintainers.

I encountered 2 licenses withing rust-windowing which bevy doesn't honour: keycodes and cursor icons.

I copied the licences and called it a day. We're adding a tiny bit of modification to winit's code by adding a reflection capability through derives, so technically we should probably disclose that, but I'm not a lawyer.

Green Tests!

Once everything was aligned, one last thing bit us: I did not notice duplicate dependency on raw-window-handle was explicitly forbidden in our deny.toml, again the community joined efforts to unblock everything!

Bevy continous tests are quite complete with different platforms, unit tests + example runs with screenshot, it's worth an article on its own.

While it gave me enough confidence in attempting such an impactful task, it's not replacing manual testing, as it wouldn't have spotted: differences in resizing behaviour, web crashing after a few frames, android compilation... But it's still continuously improving, and not at all in a bad state, game engine really are a beast to test.

"Done"

It's merged now!

From June to december 2023, that was quite a Journey!

But are we ever "done" ?

There's a good list of follow up items, drop a comment if you want to help!

I hope my story helps with understanding how open source contributions come to life, I look forward to your help ! 😉

"But didn't you want to copy paste or input '@'? How does that solve anything? What about The Altgr Problem?"

YES! But that post is long, I'm not faster than life² so I'll consider this story done, but keep your eyes open, I'm not done with open source!

❤️ Want more? Consider sponsoring me! ❤️