So there is finally some progress. Not so much on the gameplay side, but the underlying mechanics are getting fleshed out quite nicely.
As we are aiming for a SHMUP game, the client really needs to feel responsive. This will be achieved by some clever client side prediction. This will only roughly be described here. For a full introduction, I recommend you this awesome tutorial here.
Basically the server but also every client runs on it's own pace and a slow client should not block the server or other clients. This means that some sort of lock-step implementation will not work for us.
Due to network lag, the client will always be behind the server and there is basically nothing that could prevent it. The idea is that the client predicts the movement for a certain frame 60. Then it sends out the input state for this frame to the server. Simultaneously the gamestate for frame 55 has arrived (due to network lag this will always lag some frames behind).
The solution now is that the player locally stores the game state as well as the input state for the past frames.
The number of stored input frames as well as the framerate determines what input lags you can compensate.
If a mismatch between the locally computed frame 55 and the received frame 55 from the server is detected, the client code now needs to roll back its simulation to frame 55 and re-simulate the input up to frame 60. This will then be used to display the player position.
To simulate network trouble, you can use a tool like clumsy. This allows to add arbitrary lag, dropped packets, throttle, limited bandwidth or even message corruption to your in or outgoing network packets. It works for ipv4 and ipv6 and supports udp and tcp. This is very handy for testing purposes.