Using The Network Prediction Plugin In Unreal

The Network Prediction plugin is a plugin provided by Epic (still WIP at the time of writing this) to get around some of the issues that base Unreal has with accuracy focused networked games (shooters, physics based etc). This post is going to follow on from the previous post which goes over the reasons why you may want to switch to using this plugin. If you’ve not read that, i’d recommend going through it to give you a general idea if this plugin would be beneficial for your own project. In this post I’ll give a general implementation overview and also some insights into the issues that i’ve ran into when working with it.

In general what we get from implementing this plugin is a fixed network-only tick with additional buffering on both the server and client side which we can rely upon for better predicted logic. It also comes with a built-in rollback mechanism which makes it much easier to clean up mispredictions without having to reimplement that logic for each issue. The idea is to keep any logic within this plugin lightweight so that if we need to replay it due to being out of sync with the server we keep the logic contained and avoid redundant work in resource heavy systems such as animation/physics. When thinking about how to implement this plugin it’s worth considering which classes should inherit from the new network functionality as we want to keep the interconnected systems to a minimum. The plugin is unfortunately quite verbose and hard to understand at a first glance but the resulting functionality can be worth the time investment for some projects.

There is an official Readme with additional info that can be viewed on Github (but you must’ve already linked your Github account to view it). This Readme has a list of known caveats at the bottom which i’d recommend you read through before committing to it in your own project. I’d also say that it’s obvious the plugin is still not quite ready to be released and to set aside some time for digging into the plugin by yourself to better understand issues that haven’t properly been handled yet.

Enabling The Plugin

The Network Prediction plugin is not enabled by default so we need to enable it before we can use it. You can find it in the editor Plugin menu under Edit->Plugins->Network Prediction.

Or you can just manually add it to your .uproject file:

{
	"Name": "NetworkPrediction",
	"Enabled": true
}

In your projects build.cs you’ll need to add the NetworkPrediction module too:

PrivateDependencyModuleNames.AddRange(new[]
{
	..., 
	"NetworkPrediction",
	..., 
}

SimulationTick

To explain how the plugin works a bit more clearly i’m going to have to start in a different place to where you’ll be initially implementing it. Hopefully after the next few sections you’ll have a good idea of how everything connects and how to apply it to your own project. I’ll start with the SimulationTick function.

The SimulationTick is the core function of any work that you’ll be doing in the plugin. This is where all logic will be applied to your internal states at a fixed timestep. It passes in an initial state as an input and then you must pass out the output state with any changes that you wanted to make for that network frame. You’re also provided with your player input and any timestep info such as the fixed timestep ms, current frame index and total simulation time. The Output struct also includes a CueDispatch member which gives us a context which will tell us if we’re currently predicting logic or if we’re in a reconcile loop. We can talk a bit more about that later.

This function is called on all clients including simulated proxies! The input struct should always be correct on autonomous proxies and authority but it may be slightly incorrect on simulated proxies. I believe this is on the list to be updated and improved at some point.

A simple example of this that we can use throughout the post is in MockNetworkSimulation.cpp. The following code shows the Total variable being updated based on some of the input parameters.

void FMockNetworkSimulation::SimulationTick(const FNetSimTimeStep& TimeStep, const TNetSimInput<TMockNetworkSimulationBufferTypes>& Input, const TNetSimOutput<TMockNetworkSimulationBufferTypes>& Output)
{
	Output.Sync->Total = Input.Sync->Total + (Input.Cmd->InputValue * Input.Aux->Multiplier * ((float)TimeStep.StepMS / 1000.f));
}

This function belongs to a single class, FMockNetworkSimulation which is referenced elsewhere but for now the relevant code is shown below.

class FMockNetworkSimulation
{
public:

	/** Main update function */
	void SimulationTick(const FNetSimTimeStep& TimeStep, const TNetSimInput<TMockNetworkSimulationBufferTypes>& Input, const TNetSimOutput<TMockNetworkSimulationBufferTypes>& Output);
};

Sync And Aux States

The Sync and Aux states are the structs which hold our internal state. This could be a health value, a movement speed value or maybe a character related enum and we process those variables in the previously mentioned SimulationTick function. These structs are split as the sync state is intended to be used for continuously changing data whereas the aux state should be used for infrequently changing data but the functionality to actually handle these differently has not yet been implemented at the plugin level so it shouldn’t make a difference for now. The code implementing these states is available in the header file:

// State we are evolving frame to frame and keeping in sync
struct FMockSyncState
{
	float Total=0;
	void NetSerialize(const FNetSerializeParams& P)
	{
		P.Ar << Total;
	}

	// Compare this state with AuthorityState. return true if a reconcile (correction) should happen
	bool ShouldReconcile(const FMockSyncState& AuthorityState) const
	{
		UE_NP_TRACE_RECONCILE(FMath::Abs<float>(Total - AuthorityState.Total) > SMALL_NUMBER, "Total:");
		return false;
	}

	void ToString(FAnsiStringBuilderBase& Out) const
	{
		Out.Appendf("Total: %.4f\n", Total);
	}

	void Interpolate(const FMockSyncState* From, const FMockSyncState* To, float PCT)
	{
		Total = FMath::Lerp(From->Total, To->Total, PCT);
	}
};

// Auxiliary state that is input into the simulation. Doesn't change during the simulation tick.
// (It can change and even be predicted but doing so will trigger more bookeeping, etc. Changes will happen "next tick")
struct FMockAuxState
{
	float Multiplier=1;
	void NetSerialize(const FNetSerializeParams& P)
	{
		P.Ar << Multiplier;
	}

	bool ShouldReconcile(const FMockAuxState& Authority) const
	{
		UE_NP_TRACE_RECONCILE(Multiplier != Authority.Multiplier, "Multiplier:");
		return false;
	}

	void ToString(FAnsiStringBuilderBase& Out) const
	{
		Out.Appendf("Multiplier: %.4f\n", Multiplier);
	}

	void Interpolate(const FMockAuxState* From, const FMockAuxState* To, float PCT)
	{
		Multiplier = FMath::Lerp(From->Multiplier, To->Multiplier, PCT);
	}
};

As you can see both of these structs have 4 implemented functions: NetSerialize, ShouldReconcile, ToString and Interpolate. It’s important that these all use the exact spelling as there is some template magic going on under the hood and you’ll get compilation issues if they don’t match.

• NetSerialize defines how the struct should be serialized and deserialized when sending/receiving. (TIP: You can use NetSerializeOptionalValue to easily serialize Unreal type values and also reduce packet size for any defaulted values)
• ShouldReconcile defines the instructions on how to compare the clients state vs the authoritative state. If this function returns false, the simulation will revert to the most recent successful state and call SimulationTick on every frame after that until it has caught back up to the current frame. This example uses the UE_NP_TRACE_RECONCILE macro which also logs any failed reconcile data to the console when the np.PrintReconciles cvar is enabled.
• The implementation in the ToString function is used for any general debug logic of the struct.
• The interpolate function is only used when the Network Prediction Plugin simulation mode is set to interpolate. I’ve not used this much so I won’t go into it too much in this post but this allows you to define how to interpolate between your values which is called every net frame.

If you want a better example on how to use these i’d recommend having a look into MockAbilitySimulation.h.

You may have realised in the previous section that we set up the SimulationTick function with the TMockNetworkSimulationBufferTypes types which didn’t exist yet. We can define this just above the class with the input, sync and aux states like so.

using TMockNetworkSimulationBufferTypes = TNetworkPredictionStateTypes<FMockInputCmd, FMockSyncState, FMockAuxState>;

The Input Struct

The input cmd struct is the final piece of the puzzle for the SimulationTick function. This is where player input will be filled and sent to the the simulation for us to affect any state values. The MockNetworkSimulation header has the following setup:

// State the client generates
struct FMockInputCmd
{
	float InputValue=0;

	void NetSerialize(const FNetSerializeParams& P)
	{
		P.Ar << InputValue;
	}
	
	void ToString(FAnsiStringBuilderBase& Out) const
	{
		Out.Appendf("InputValue: %.4f\n", InputValue);
	}
};

To keep things simple it has a single InputValue member. It also shares the NetSerialize and ToString functions from the Aux and Sync states and the functionality is identical. The other two missing functions from the states do not need to be implemented in this struct.

You may be wondering how we send the player state to the simulation. This is where we start implementing functions into our Unreal class (and we will go further into how to do that in the next section). The relevant function for this is the ProduceInput function. This is called once every network tick on the controlling client before the SimulationTick is called to make sure that the input is always as up to date as possible before processing it. If the controlling instance is not the authority it will then send this packet to the authority for it to process the SimulationTick with the same data. It’s important to mention that for every input packet the latest 6 frames of input state is sent which ensures that the server can always see the full history of input if we fail to send any packets so this input struct should be kept as small as possible. It should also represent the player input as close as possible, this packet is easy to alter when cheating so it may also be worth verifying it before processing it.

You can see in the mock example that they’re just setting the input value based on a global bool.

void UMockNetworkSimulationComponent::ProduceInput(const int32 DeltaTimeMS, FMockInputCmd* Cmd)
{
	if (MockNetworkSimCVars::DoLocalInput)
	{
		Cmd->InputValue = (MockNetworkSimCVars::RandomLocalInput > 0 ? FMath::FRand() * 10.f : 1.f);
	}
	else
	{
		Cmd->InputValue = 0.f;
	}
}

Hooking It All Up

At this point we’ve gone through how to define the actual state and functionality of the Network Prediction layer but not how to actually register the tick with the subsystem and process any of the functions. This process is much easier if you’re able to inherit from the UNetworkPredictionComponent class. If you can inherit from this class your starting point to copy from will be the UMockNetworkSimulationComponent in MockNetworkSimulation.h. I’d recommend placing all NP logic inside a component class rather than an actor class just to simplify this part as much as possible. If not you’ll have to work out how to implement this into an actor as the creation order and functions to override will be slightly different.

First of all we need to create a model definition for our new network model. This is defined at the top of MockNetworkSimulation.cpp. The simulation should point at your network model class, StateTypes at the previously defined buffer types and your driver is the Unreal object which the functionality is going to be implemented in. In this example it’s the UMockNetworkSimulationComponent that is what is inheriting from UNetworkPredictionComponent. You can also use GetSortPriority to define which order you want each model to tick in to ensure your logic is always resolving as you’d expect it.

class FMockNetworkModelDef : public FNetworkPredictionModelDef
{
public:

	NP_MODEL_BODY();

	using Simulation = FMockNetworkSimulation;
	using StateTypes = TMockNetworkSimulationBufferTypes;
	using Driver = UMockNetworkSimulationComponent;

	static const TCHAR* GetName() { return TEXT("MockNetSim"); }
	static constexpr int32 GetSortPriority() { return (int32)ENetworkPredictionSortPriority::Last; }
};

NP_MODEL_REGISTER(FMockNetworkModelDef);

Once that has been set up we can look at setting up the simulation object. The header defines this as a public member variable using the simulation class which we previously defined in the definition:

// Our own simulation object. This could just as easily be shared among all
UMockNetworkSimulationComponent TUniquePtr<FMockNetworkSimulation> MockNetworkSimulation;

We then need to create it in the InitializeNetworkPredictionProxy function which is overridden from the base class. InitializeNetworkPredictionProxy gets called during UNetworkPredictionComponent::InitializeComponent() which is only called once when the component has first been initialized. If your network driver is an actor you’ll have to find an equivalent of this function:

void UMockNetworkSimulationComponent::InitializeNetworkPredictionProxy()
{
	MockNetworkSimulation = MakeUnique<FMockNetworkSimulation>();
	NetworkPredictionProxy.Init<FMockNetworkModelDef>(GetWorld(), GetReplicationProxies(), MockNetworkSimulation.Get(), this);
}

If you ever need to pass something into the simulation instance, maybe a pointer to another system then this function/pointer is how you’d do it. Calling Init here on the network model is how it is registered and setup with the UNetworkPredictionWorldManager which is the subsystem that handles the management of all network simulations. This should all be easy enough to copy and paste into your own code so far but there are still a few more steps to look at.

We’ve also got 3 functions to implement in the component, one of which we’ve already seen. This is the InitializeSimulationState, ProduceInput and FinalizeFrame. InitializeSimulationState is called once before any SimulationTicks have been called to give you a chance to set up the Sync and Aux states with any default values. ProduceInput we’ve gone over previously and FinalizeFrame is a function that is called once per Unreal frame so that we can read data from the Sync and Aux states and then use them in the Unreal simulation. As we’ve fixed the network tick to a static timeframe it means that we may get more than one SimulationTick for every Unreal tick we get. This function ensures that we’re getting the correct data at the right time, avoiding all chances of grabbing data from the simulation while a rewind is in progress. Unfortunately these functions are not overriding anything, you’ll just get a nasty compilation error if you forget them.

	// Seed initial values based on component's state
	void InitializeSimulationState(FMockSyncState* Sync, FMockAuxState* Aux);

	// Set latest input
	void ProduceInput(const int32 DeltaTimeMS, FMockInputCmd* Cmd);

	// Take output of Sim and push to component
	void FinalizeFrame(const FMockSyncState* Sync, const FMockAuxState* Aux);

There’s also a secret fourth function which can be useful to know about (it’s not really secret but took me a while to find it…). It’s the RestoreFrame function and you an see an example of it in CharacterMotionComponent.h. This function is called before restoring to an old frame when a resimulation has been triggered which i’ve needed to use before to compare the difference of the before and after states. This function is optional though so feel free to leave it out if not needed.

void RestoreFrame(const FCharacterMotionSyncState* SyncState, const FCharacterMotionAuxState* AuxState);

Finally the last bit of implementation for the component is to override a few defaults in the constructor. These ensure that the replication is enabled and that InitializeComponent is called in the base class which is required to call the InitializeNetworkPredictionProxy function that we inherited from earlier on.

UMockNetworkSimulationComponent::UMockNetworkSimulationComponent()
{
	bWantsInitializeComponent = true;
	bAutoActivate = true;
	SetIsReplicatedByDefault(true);
}

If you’ve followed everything up until now and you’re finally getting functionality in your SimulationTick called, well done! There’s a lot to add to get things running and also a few places where it can quietly break. If you’ve not managed to get it working i’d recommend breakpointing the InitializeForNetworkRole in UNetworkPredictionComponent. It may be that the ownership of your component is wrong as we’re only able to predict logic on a component that we’re an autonomous proxy for. If the ownership is not set up correctly you may only see the SimulationTick being called on authority and it skipping all clients.

If Your Driver Class Cannot Inherit From UNetworkPredictionComponent

Sometimes you’re not going to be able to inherit from UNetworkPredictionComponent if you need to use another Unreal component type such as the Character Movement Component. This is where things get a little more ugly to implement as there is a lot more to copy and paste. You just need to pull the relevant functionality out of the UNetworkPredictionComponent class and into our own class. I won’t go through this in detail here as it will double the length of the article but it’s important to reimplement each FReplicationProxy as it has been in the header file and then make sure you’re setting up the replication functionality identical to how it has been setup in the cpp. It’s up to you if you want to also expose the virtual functions but I probably would to keep it as consistent as possible.

The NetworkPredictionComponent class hasn’t changed within the last year but if Epic do start working on it again it’ll be important to make sure you update your own code with the new changes too so that you’re keeping in sync with the plugin and any potential bug fixes.

Using NP Cues

Cues are the last major piece of the puzzle in the Network Prediction plugin. If you’ve ever worked with GAS they have a similar purpose; events that can be triggered to show VFX and play audio at the correct time. However the cues in NP are slightly more powerful as we can also choose if we want cues to be resimulated and which replication targets should play the cue. This is an example of a minimal cue in MockNetworkSimulation.h.

// A minimal Cue, emitted every 10 "totals" with a random integer as a payload
struct FMockCue
{
	FMockCue() = default;
	FMockCue(int32 InRand)
		: RandomData(InRand) { }

	NETSIMCUE_BODY();
	int32 RandomData = 0;
};

// Set of all cues the Mock simulation emits. Not strictly necessary and not that helpful when there is only one cue (but using since this is the example real simulations will want to use)
struct FMockCueSet
{
	template<typename TDispatchTable>
	static void RegisterNetSimCueTypes(TDispatchTable& DispatchTable)
	{
		DispatchTable.template RegisterType<FMockCue>();
	}
};

It’s also required to register the cue in the cpp like so:

NETSIMCUE_REGISTER(FMockCue, TEXT("MockCue"));
NETSIMCUESET_REGISTER(UMockNetworkSimulationComponent, FMockCueSet);

It’s also necessary to set up the function which will get called when the cue is received. The cue type must match the struct that has been created and the parameters input gives us some other nice functionality such as how much time has passed since the cue was invoked.

void UMockNetworkSimulationComponent::HandleCue(const FMockCue& MockCue, const FNetSimCueSystemParamemters& SystemParameters)
{
	UE_LOG(LogMockNetworkSim, Display, TEXT("MockCue Handled! Value: %d"), MockCue.RandomData);
}

Once you’ve defined the cue you can call Invoke in the SimulationTick to call that cue. This is where you would also pass in any additional data through the constructor of the cue.

Output.CueDispatch.Invoke<FMockCue>( FMath::Rand() % 1024 );

This cue is really basic though and NetSerialize has not been set up to work so in practise it should only work for non replicated cues. There’s a much better example in MockAbilitySimulation.h where both NetSerialize and NetIdentical have been implemented. NetSerialize is exactly the same as the previous structs that we’ve set up and NetIdentical is similar to the ShouldReconcile function where it returns out if the cue is identical to the previous cue or not. This is only relevant when the Trait of the cue has been set to a type which will resimulate.

// Cue for blink activation (the moment the ability starts)
struct FMockAbilityBlinkActivateCue
{
	FMockAbilityBlinkActivateCue() = default;
	FMockAbilityBlinkActivateCue(const FVector& InDestination, uint8 InRandomType)
		: Destination(InDestination), RandomType(InRandomType) { }

	NETSIMCUE_BODY();

	FVector_NetQuantize10 Destination;
	uint8 RandomType; // Random value used to color code the effect. This is the test/prove out mispredictions

	using Traits = NetSimCueTraits::Strong;
	
	void NetSerialize(FArchive& Ar)
	{
		bool b = false;
		Destination.NetSerialize(Ar, nullptr, b);
		Ar << RandomType;
	}
	
	bool NetIdentical(const FMockAbilityBlinkActivateCue& Other) const
	{
		const float ErrorTolerance = 1.f;
		return Destination.Equals(Other.Destination, ErrorTolerance) && RandomType == Other.RandomType;
	}
};

This cues Trait is set to Strong which as you can see in the comment below will be shown as correct as possible. This is much more expensive due to needing the identical checks so there are a lot of other options of traits for cues which should be used most of the time. A strong cue should only be used for any logic which could affect the gameplay in any way. There are other traits which can be used in NetworkPredictionCueTraits.h and you can always define your own struct with your own rules if these don’t fit your needs.

	// Invoked and replicated to all. NetIdentical testing to avoid double playing, rollbackable so that it can (re)play during resimulates
	// Most expensive (bandwidth/CPU) and requires rollback callbacks to be implemented to be correct. But will always be shown "as correct as possible"
	struct Strong
	{
		static constexpr ENetSimCueInvoker InvokeMask { ENetSimCueInvoker::All };
		static constexpr bool Resimulate { true };
		static constexpr ENetSimCueReplicationTarget ReplicationTarget { ENetSimCueReplicationTarget::All };
	};

Working Between Systems

It is possible to read or write into the simulation from outside of the functions that we implemented by using NetworkPredictionProxy->ReadAuxState/WriteAuxState or the Sync state equivalents. You have to be extra careful here when writing to the simulation as it can be less obvious which frame you’re currently in due to the different tick rates of Unreal/NP ticks. Writing to a state is going to mean that variable is changed for the next SimulationTick. This is how you’d expect to read and write between systems within NP but i’d recommend adding a NetworkSimulation sort priority with GetSortPriority() so you can ensure correct order and keeping any writes outside of NP to a minimum.

void UCharacterMotionComponent::SetMaxMoveSpeed(float NewMaxMoveSpeed)
{
	NetworkPredictionProxy.WriteAuxState<FCharacterMotionAuxState>([NewMaxMoveSpeed](FCharacterMotionAuxState& AuxState)
	{
		AuxState.MaxSpeed = NewMaxMoveSpeed;
	}, "SetMaxMoveSpeed");
}

void UCharacterMotionComponent::AddMaxMoveSpeed(float AdditiveMaxMoveSpeed)
{
	NetworkPredictionProxy.WriteAuxState<FCharacterMotionAuxState>([AdditiveMaxMoveSpeed](FCharacterMotionAuxState& AuxState)
	{
		AuxState.MaxSpeed += AdditiveMaxMoveSpeed;
	}, "AddMaxMoveSpeed");
}

Using The Tick Context

Sometimes when you’re working in your SimulationTick you’ll need to define logic based on if you’re currently predicting or reconciling. The enum to check this is slightly hidden but you can find it within the CueDispatch context.

if (Output.CueDispatch.GetContext().TickContext == ESimulationTickContext::Resimulate)
{
  // Do logic
}

Where To Find Examples

Your best source of documentation is the NetworkPredictionExtras plugin as that has a few different examples of how you’d implement fairly simple systems. MockNetworkSimulation is a great starting point and that’s why most of the examples in this article use it. The MockAbilitySimulation class is a good example in how to implement a basic ability that affects different stats on a character and some more informative cue examples. I’ve also found use in FlyingMovementSimulation and CharacterMotionSimulation/CharacterMotionComponent when thinking about how to integrate the system with components or character movement.

Issues I’ve Faced

Not Having Enough Access To Data Outside Of The Simulation

There have been times during development where I needed to know the current frame index that the Network Prediction plugin is currently at so I could sync functionality between Unreal and NP timelines. By default this isn’t really available so to get around this I had to expose FFixedTickState in the UNetworkPredictionWorldManager. This gives you access to some other features such as the last confirmed frame and the frame offset between the client and server timelines. I posted this as a request on UDN months ago but it still hasn’t made it’s way into the codebase.

const FFixedTickState& GetFixedTickState() const { return FixedTickState; }

Physics Syncing Is Still Broken

There’s a lot of code dotted around suggesting that there has been some work put into properly syncing physics objects. Sadly this has been left in a state where if it gets enabled in any way then it’ll break the rest of your simulation as there is a check(false) in the UNetworkPredictionWorldManager::ReconcileSimulationsPostNetworkUpdate_Internal if bDoPhysics is true. The simulation will set up as being physics enabled if any of your network models include a PhysicsState definition or override FGenericPhysicsModelDef. For now i’d avoid using this but i’d love to see Epic come back to this and fix it up again in the future.

All Or Nothing Approach

Due to the buffered packets when the server received input from the autonomous proxy and then another buffer when clients receive data from the server there is always going to be a large delay when reacting to players input compared to using Unreal RPCs. If everything is built within the NP plugin, this is fine as you can predict as needed and work around these issues. When you’ve got systems built in default Unreal you no longer have these delays though which means interactions will trigger earlier than when they would inside NP. It’s very easy to get stuck in a place where you’ve introduced new timeline related issues into your workflow as you need to think about this delay more often. I’d personally recommend an all or nothing approach. Either fully commit to using the NP plugin or avoid it and make do without it.

It may be hard to commit to an all in approach due to Unreal systems such as the CharacterMovementComponent and GAS not using this plugin. Epic have assured me that work on the CMC is happening to bring these together and i’m hoping that will be in the Mover 2.0 update that’s coming soon. GAS is not on the roadmap to share this timeline though so the best approach would be to build your own system.

Limited Documentation And Forum Posts

It’s quite obvious that this plugin is still in its early days (it’s the reason I started these blogs!). There is very little available documentation, most of what i’ve found is available on UDN but only those working at companies will get access to that. Be ready to dig in deep to find out where your issues are coming from as it’s a fairly common occurance. The plugin is powerful and in my opinion, extremely high priority for Epic to be solving these issues but it still has a way to go.