What are your thoughts on anti-cheat software that has access to your computer while you play? Blizzard implemented it with Warden, and Bethesda just implemented it with Doom. It seems like more and more games are going to this to stop people from cheating in their games. I think that if we want to keep the game cheat-free it's almost necessary, but I can see why people would not want a game to know what you are doing on your computer outside of the game.... like loading a map hack that looks at your game's memory.
Jason Weinmann puts it best "Never trust the client". This basically means that the server will always be checking and verifying that the client is not lying (players cheating).
I really do not expect to see any client-side anti-cheat implemented because there won't be any need as the server will be handling everything.
What do you mean by access? Because typically hacks are catergorized via the CPU privelage Ring system Ring0 being scripts executed in the Kernel then you have Rings 1 and 2 which are used for device drivers and Ring3 being for applications. Anti cheat methods pretty much revolve around Ring0 and Ring3. An example of a Ring0 anti cheat is Battleye and Valorants anti cheat, which have stirred up controvery over the years particularly because they operate inside the kernel which can bring a whole host of issues to your OS.
What are my thoughts on how Pantheon should approach this?
My stance is that Pantheon if it is to have a form of anticheat it should not be a Ring0 anti cheat and should only be operating whilst the game client is open. A good example of why I am against it operating at all times and have full access to your PC is Valorants Anti cheat which has gotten alot of bad press due to the fact that it begins operating before your even logged into your computer and people have reported crashes, applications refusing to open and some applications randomly closing.
What I would like to see is a more focused attempt by either the mod team and/or their anti cheat to prevent botting as its usually botting that plagues MMOs rather than Aim assistance, ESP, movement assistance for First person shooters.
bigdogchris said:Jason Weinmann puts it best "Never trust the client". This basically means that the server will always be checking and verifying that the client is not lying (players cheating).
I really do not expect to see any client-side anti-cheat implemented because there won't be any need as the server will be handling everything.
Brooks said:bigdogchris said:Jason Weinmann puts it best "Never trust the client". This basically means that the server will always be checking and verifying that the client is not lying (players cheating).
I really do not expect to see any client-side anti-cheat implemented because there won't be any need as the server will be handling everything.
I don't think that's possible. There will always be some work handled by the client-side simply because they would need a supercomputer to process it all and the data between the servers would be significantly more to transmit all that data. There are just too many calculations going on with things like movement, bots, gold production, radar for rare spawns, map hacks... all of this and way way more is typically handled by the PC and transmitted to the servers when done.
https://youtu.be/jo4unYGQGB0?t=485
Whilst I'm not paranoid to believe client anti-cheat software is any kind of spyware, I do think they often cause more trouble than they are worth.
Also, Jason sounds a more than competant dev and is convinced they can do enough off the client (or at least enough *checking* off the client) to stop cheats.
Brooks said:... There are just too many calculations going on with things like movement, bots, gold production, radar for rare spawns, map hacks... all of this and way way more is typically handled by the PC and transmitted to the servers when done.That's fundamentally not how MMO's work or are implemented. If the client did all that, all of that would be exploited. Everything the client is authority for, gets exploited. That's why you only send what's absolutely necessary, and fully expect that will be exploited.
MMO's have had server side validation for all important transactions since they have existed.
Every single one that does hasn't been (and continues to avoid being) exploited.
Every single one that does NOT has been (and continues to be) exploited.
It's up to the devs how much validation they want to impose. That BSC video is really good at showing what's possible with modern servers.
Do you validate every movement? You probably should, but you could get away with a temporal ratio if things got really busy.
Do you validate every combat action and non-movement keypress? Yes.
Do you validate every transaction between players? Hell yes.
"But, but but.. EQ1! ShowEQ!! MySEQ!!" , you say and yes, it's an alternate view of the same information everyone is seeing. That's all information provided to the client, shown in a different way so as to provide situational awareness to the player. Every single thing that's provided is, by design, required to render the scene accurately. It's just location and vector, location and vector, for every actor/mob/player/whatever in the zone. It also happens to be things like the name of the actor, mob, player, whatever which, because it's sometimes very important to know if a name(d) creature is in a zone, anywhere, at all, provides you with an advantage.
Unity could be written to provide these things to the client in the same way. Unity can also not provide things things, and provide updates in a different way.
Presumably, the "industry veterans" developing Pantheon would take steps to ensure the value of 'MyPanth' would be extremely limited, by limiting what is sent from the server to the client is again, the absolutely minimum required to render the scene.
It's also why latency is such a ***** in MMO's. If you're dealing with a 200-500ms one way packet delay, every validation takes 400-1000ms (0.4 to 1.0 seconds) PLUS server processing time.
Yet, if you have a nice reasonable 50-90ms one way latency (most of north america, from B.C. to Florida for clients, with a server in, say, Nebraska) then 100-180ms plus server time is perfectly fine, provided things are working well.
If you have regional servers (West Coast, Mountain, Central, East Coast) and people are strongly encouraged to play in their time zone, then you can make things downright reasonable.
But it's also why, when you start talking about mechanics, like, in-game scripts where players are expected to "react" within 250ms to an event, like a state or trigger, or alert, or tell, or wind-up, or FIRE ON THE FLOOR, you're making it impossible for people in Anchorage to react when your server is hosted in Orlando. Physics forbids it, even if your server processing time is sub-millisecond.
Also, in reality, you don't get to choose Nebraska, you get to choose Chicago, and hope it's good enough for most players within a certain distance.
Part of the challenge is that modern MMO's have an enormous view distance. In some cases, in outdoor zones, over 1000m. Well, crap.. now what? The client visibility is so massive, and people can see a_rabid_bear_06 from 800+ meters away. If they can see it, they expect to be able to click on it and/or tab target to it. If they can see it but can't click or interact, that's a non-immersive experience.
Some abilities need to work over such massive distances. Pet commands, for example. So, it's not like you can adjust the visibility of the mobs in realtime based on the viewport/spin-rate of the client. That would be absurdly inefficient. Result? The client has, in it's memory, a visibility 'bubble' of whatever distance the developers decide is appropriate for their expected latency and client rendering performance. That bubble can be turned into a real-time radar, and there's absolutely nothing developers can do to stop that. The client requires it. It needs to be in memory. As such, it can be read and rendered in a different way.
If you've ever watched the TV series Seal Team, you'll be familiar with them often having a drone above their active operation. This provides them with the same environment, rendered in a different way. It's a different perspective of the same common shared environment. It's also an astonishing tactical advantage.
This is clearly seen by the advantage such a simple thing offers players using MySEQ and similar utilities in MMO's.
It's all the same data everyone else is getting, they're simply rendering it in a way to gain a tactical advantage. It's also called cheating, by some. :)
It's important to understand that a subset of any target demographic will consider ALL the data the server is sending to them to be "part of the game" and within scope for their use as a paying customer. Put another way, if the developers don't want the client (and thus, the player) to have that information? Then don't send it to the client.
In any case, server side validation of important transactions has been and is a solved problem for 20+ years.
You're talking about two different things. Movement is not a 100% server side validated action in EQ1. Never has been, likely never will be.
It COULD be. The developers could do it. They choose not to. They continue to choose not to.
Yet, transactions with NPC's are server side validated.
Combat actions are server side validated. Spell casts, actions, trades, all of that. Server side validated.
The current and past developers of EQ1 made the conscious choice NOT to validate every movement action in EQ1 with the server. The EQ1 client is the authority for some movement, and the result is: movement is exploited.
Watch the BSC video. It doesn't have to be that way. It's a choice.
Games made before and after EQ1 have/had server side movement validation, in addition to other actions performed by the client.
Ultimately, why developers make these choices is because of the "cost" in latency. If you validate every client movement, that's costly. Generally, developers don't see the value in doing this, when there are other things in the outstanding bug list that are vastly more important. Yes, it's important. Yes, it should be done. Yes, it can be done. Yet, it often isn't.
I am not a programmer and have very limitted understanding of how any of this would work. But to me that's beside the point...HOW it would work doesn't matter. IF they can get something to work I would definitely support it. Even if it was something that ran on my computer in the background while playing the game.
If they can not get it to work I've come to the understanding that people out there will cheat/exploit/hack and I just have to do my best to not let it ruin MY experience in the game.
Brooks said:I don't think that's possible. There will always be some work handled by the client-side simply because they would need a supercomputer to process it all and the data between the servers would be significantly more to transmit all that data. There are just too many calculations going on with things like movement, bots, gold production, radar for rare spawns, map hacks... all of this and way way more is typically handled by the PC and transmitted to the servers when done.
The client is responsible for rendering what you see on screen, but that doesn't mean that the server see's the client as authoritative to what your character is doing.
Additionally, the connection to the game will be encrypted so man-in-the-middle attacks will be less likely to read/manipulate data. The game server is only going to send the necessary information to the client. So enemy position for example won’t be sent to the client to exploit if the player is far enough away that it doesn’t matter to the PC. NPC loot won’t be shown to the client (or even server for that matter) because it won’t be generated until the NPC is killed. Systems like this are improvements over the way things were done in EQ and make it less likely to be exploited.
While not as simple as this explanation, the client will only receive the information needed to display things on it. Think of your client as a dumb-terminal that just adds graphics to the data the server sends it. Anything your client sends back to the server as an action will be checked to verify it that's even possible to be done by your charcter, and if not, it will be denied and logged for further investigation.
Reading this I thought, "Oh no, I now feel the urge to have to explain this in detail and it's 3AM and I have to be up in 5 hours. Thanks for the detailed and concise explanation, Vjek.
Vjek is correct in that this was something dealt with long ago, but under a few limitations brought on either by capability or the limitations of the network medium. Simply running a similar authentication rubric as EQ with the inclusion of a few noted features would handle 99% of the exploitation and cheating. Remember, with everything in this area it's about volume. You want to reduce the total volume of infractions so the hands-on GM team have fewer cases to work with directly.
You also want the foundation of a client and server archtecture to be somewhat malleable. This allows for designing new tools from which to deal with new issues. EQ's Client wasn't exactly known for that but understandable given the era.
Over the years, many people have wondered why EQ1 doesn't (and didn't) perform server side validation of all movements, teleports, and movement rates.
(IMO)
In short, it's mostly because of three things; Selo's, Teleports, and clickies.
Group effects, such as having a bard in the group with run speed improvement and levitation affects the character's movement rates without them personally being able to do that. It also places them in very strange Z locs, compared to 'normal'.
So, on one hand, you could (at an extraordinary asynchronous load cost) iterate through an entire character's inventory, spells, items, abilities, songs, etc to determine if any of them contain anything like an effect that will affect the movement rate of the player, or give them the ability to levitate or teleport. You could do that. But, if they're in a group, now you have to do that for every member of the group, as part of the validation.
That... is not a trivial task. It takes an unknown amount of time, based on the volume of items each character in the group is carrying. Some might have jboots as the last item in their inventory, but it affects their movement rate, so.. you can't ignore it. You have to check them all. While servers are fast, and in-memory compares are also fast, it's still 'costly' in CPU terms.
While it's true you could take a shortcut and simply enumerate and compare the effects current on the player, that doesn't work for teleportation, as it's instant, and carries no temporary residual effect/indicator/icon. If teleportation always did have or show an effect (even hidden, for 1-2 ticks after), then that method would work for most things (like jboots, selo's, lev, sow, and teleportation). You could handle most situations, at least for teleportation and movement rate exploits.
It would mean if that player moved more than it's expected movement rate/distance in a single tick (while not falling), yet didn't have a teleport effect on them? Possible movement/teleport exploit.
If these numbers were 2, 3 or 10 times the expected values? Definite movement/teleport exploit. (this would be a very fast server-side check, even in a full group)
So, my personal opinion is, faced with the daunting task of adding an unknown, yet non-trivial / significant amount of server-side load to every movement check, (and not having the framework to catch everything in 1999) they simply opted for the easy route, and voila, movement/teleport/speed exploitation in the past and present for EQ1.
vjek said:So, my personal opinion is, faced with the daunting task of adding an unknown, yet non-trivial / significant amount of server-side load to every movement check, (and not having the framework to catch everything in 1999) they simply opted for the easy route, and voila, movement/teleport/speed exploitation in the past and present for EQ1.
The modern "norm" for this is to have a separate server cluster that is responsible only for client validation and that operates in parallel with the regular game processing. That load is "easily" spread out and handled by having different servers/clusters for each purpose. While there is a small degree of overhead in distributing client input to multiple servers/clusters and collating the results, there's no need to do everything on just one server even if that's how it appears to the player.
I think they will exist in Pantheon because the developers will make the same choices all those other MMO devs you listed have made, namely, that it's too costly (latency + CPU) to validate every movement, server-side.
But it doesn't have to be that way, and the technology exists to entirely prevent these exploits.
Essentially, most MMO developers make the conscious choice that players exploiting movement "isn't that bad".
Personally, I think it's worth validating some movement, server side. Not all, but some.
But in none of the above would I personally want anti-cheat software running on my PC. Not only is it not required for anything EXCEPT movement exploits (because you can do all the validation on the server for everything else), the negatives of having it present, in whatever ring, vastly outweigh the marginal possible benefits.
I don't believe ring 0 should ever be used for a video game (outside of F2P games) There will always be ways to catch botters and multiboxers without it. Almost everything should be server side or cheating will be the craziest thing in the game, like god mode. Cheaters should not be forgiven but banned. Too many games have it as a slap on the wrist. Let the cheaters buy the game again, milk cheaters for money.
You know you are doing something wrong when you use a program to play the game for you.
You know you are doing something wrong when you use a program to let your character run faster or warp around.
I would volunteer to be a "guide" to only find botters and speed hackers, community service at its best.
vjek said:Over the years, many people have wondered why EQ1 doesn't (and didn't) perform server side validation of all movements, teleports, and movement rates.
(IMO)
In short, it's mostly because of three things; Selo's, Teleports, and clickies.
And that when EQ1 released it was being accessed by users over slow, high-latency dial-up modems. There were concerns that failing to limit the quantity of checks being transfered would impede gameplay for a notable percentage of users.
Applying the thinking of today, to games coded 20+ years ago is kinda crazy. Especially if you were a kid then. Or not even born.
EQ1 was developed back when the internet was just starting to be a thing.
Most people were still dialing into BBS's and high-speed was a 14K baud rate modem.
A 1280x1024 monitor was to die for. Hard drives were magnetic and 5400rpm at best. RAM, if Gates (the tech thief) was to be believed, unnecessary.
And if you built your own PC, you spent days trying to get the audio drivers working. (long before audio was built into MB's.)
Thoughts of security were fleeting at best for programmers. Most were still trying to understand the new concept of programming and it's daily changes. The biggest struggle at the time was to pass the necessary amount of data through small dial-up speeds.
I started in late 98' in the EQ1 beta. I jumped from UO as soon as I got the invite.
UO had been new and exciting when I started it in 96' but the exploits (not hacking) and bugs were pervasive and anathema. In later years, when I worked with UO peeps in the industry, they told me the original UO DB was all flat file. No relational DB was really available at the time.
PC and internet tech exploded starting around 1999 or so. After everyone realized Y2K wasn't going to destroy the world. It could be argued that if EQ1 development had started a couple years later than it did, it's history would be much different. But then I guess that's what Vanguard was.
I feel that client side anti-cheeting software is an invasion. There is no guarantee that it is just collecting game data, we have no control over what goes in and out. There is also the potential for this software to be hacked and/or tested for its own exploitable features that could be re-used against the game.
Server side checking is non-invasive and real-time checking and historic testing is easily achievable too.
Jobeson said:You know you are doing something wrong when you use a program to play the game for you.
You know you are doing something wrong when you use a program to let your character run faster or warp around.
Totally agree, cheating is a concious decision. People know when they are cheating (eventhough cheaters would never admit to it). I think botting and the like, once detected, should be warned once, then the account be removed on a second infraction. The game suffers from such people being allowed to continue such activities.
Jobeson said: Unless your warning is a month long ban it is simply promoting cheating until caught.
That's like saying leaving your car unlocked is promoting theft. I agree that breaking the rules should have stern penalties, but you can never blame the person or company that was wronged and you can never say "it's ok you stole, the store manager was in the bathroom, it's his own fault."