Benchmark Result Discussion

[MacGruber]

The 100% resolution per eye for the Vive Cosmos should be 1440 x 1700 pixels (2880 x 1700 pixels combined). Is that what your setting is?

VR headsets usually run with slightly more than their native resolution. E.g. my own Oculus Rift CV1 runs at 2688x1600, although having only 2160×1200 actual resolution. That is normal and used for ASW/MS, apparently even when that is turned off. Running at 6912x4092 of course is not normal, when your native resolution is only 2880x1700.

 

[honda78]

Thanks taichi, the test above was already done on a fresh download of VaM.
I found some never AMD chipset drivers for Windows 11, reverted from SLI to single GPU, used fixed CPU Clock of 4Ghz, moved the directory from SSD to NVMe, tried fullsceen and with a Rifts S, still same results.
It looks like i am the only one who tried the benchmark with a Threadripper, so maybe i'm simply out of luck with my specific hardware.

View attachment 76267
View attachment 76268

A reply to self: Had the time to reconfigure my PC -> disabled all virtualization (Hyper-V, VBS...) and changed memory access from distributed to local. Physics time in baseline3 is still too high but 20 FPS is a big step up from 3 single frames a second. Seems to be a problem with the CPU scheduler somewhere and VaM is the only software i came across that behavior - but i'm certainly no the right guy to elaborate about that.
Still though @meshedvr - it could be worth to check a future VaM release on a threadripper, if you happen to have one at disposal...

 

[HolySchmidt]

Ok, here we go:



So, what do we have?

Picture 01: 9700K stock / 3080 FTW3 stock / RAM 3400 CL14
Picture 02: 12700K stock / the same

The result is nearly the same ... why is that? Pretty simple: I was bottlenecked by my GPU the whole time, 97%+ workload all the way through. Monitored it with MSI Afterburner + Riva Statistics Tuner + HWInfo.
But here's the twist: I saved 2 scenes before changing my CPU + motherboard and changed nothing to it after loading it with the new system, not even the camera angle. 1:1 comparison.

(2560x1440 in desktop)

Result:


Scene 01: 2 persons (animation on the female, some bouncing up and down), 1 simulated clothing item, 3 lights, 1 mirror + environment
Before: ~64fps (GPU workload around 70%)
After: ~97fps (GPU workload around 98%)


Scene 02: 3 persons, 3 lights, no clothings, simple wooden floor, animation on the male, indirect movement on both females
Before: ~31fps (GPU workload around 40%)
After: ~52fps (GPU workload around 65%)

That is an increase in real-game footage of around 50%! That is massive! I can't see that improvement in the benchmark. And as much as I do like the effort and I really appreciate the work that went into it (I paid one month patreon just for this to macgruber), it makes no sense to me. Like I said before: there has to be more physics-stuff going on. Right now it's more like a graphics benchmark.

I mean a good benchmark for a game that is both, CPU and GPU demanding, has to test both. So therefore I made my own "benchmark" that gave me more information on the real performance when playing. And I can say this is huge for VaM performance. And this proves to me without a doubt, like I think Jiraiya already said: CPU performance on this is extremly important. I can only advise to anyone wanting to improve the performance in VaM: compare the single core performance of CPUs.
Looking at this: https://cpu.userbenchmark.com/Compare/Intel-Core-i7-9700K-vs-Intel-Core-i7-12700K/4030vs4119
... is nearly what I got as an improvement. It says around 40% more single-core, I got around 50% more fps.

Of course I made all testing in 1440p (fuck, I forgot to make a testing on my old system in 1080p with some scenes ). And the lower the resolution the more your CPU has to work. Which also means that with all those VR-headsets having a high resolution the improvement will decrease (fuck again, forgot to make some VR-testing-scenes ).

btw: I don't wanna offend no one with "showing" my new hardware. I also said I under no circumstances would buy Intels 12-series. I lied
I thought you have to install Windows 11, which is wrong. I'm using Windows 10 right now and I will make further tests by disabling all e-cores since I read this will be even better with gaming.
If anyone in here wants me to test anything, whatever, pm me.

I'm gone for some testing right now. First VR use with the new one, I'm excited.

 

[Jiraiya]

Those results are not the same.
Benchmark 3 clearly shows a marked difference in physics time and FPS.
People seem to be putting far too much stock in the one big number in green and not checking the rest of the stats.
You can see your CPU based numbers have all improved.

 

[HolySchmidt]

Yeah I know, but the point is that the result, and that is what most of the people have a look at, is nearly the same. Because in the end it's all about fps of course. And that didn't change really.

And what I experienced so far ingame is way more than the difference between 78fps and 90fps in Baseline 3.

 

[Jiraiya]

Yeah, your point seemed to be that the benchmark didn't reflect "real game performance" as much as it should.
That big number is just an average of lots of averages, not a huge comparative indicator of specific changes.
The physics heavy benchmark 3 does show the CPU difference, but your "old" CPU wasn't 'that' bad anyway. I get what you are saying about your scenes showing a bigger improvement from the upgrade than the benchmark indicated.

I think this was more of a way for people to compare the same hardware to know if they fucked something up. "Hang on, you have the same CPU as me, why is my FPS half yours?" or to reassure people that thought they had bad frame rates that it's just how this game works at this point with the old version of Unity.
Posted results also give people looking to buy hardware guidelines of what sort of thing to expect.
If you want detailed comparisons you need to dig into the smaller numbers and ignore the big green one.

 

[HolySchmidt]

I understand what you're saying and it makes sense, nonetheless, if I have an improvement of around 50% in fps ingame but the "score" of the benchmark does not reflect that, then it makes no sense to me. Cause all the miliseconds and timings you get as a result is just some numbers. That's why I wanted to see how the actual fps ingame compares to the fps result in the benchmark. And there's a huge gap.

 

[keycode]

I understand what you're saying and it makes sense, nonetheless, if I have an improvement of around 50% in fps ingame but the "score" of the benchmark does not reflect that, then it makes no sense to me. Cause all the miliseconds and timings you get as a result is just some numbers. That's why I wanted to see how the actual fps ingame compares to the fps result in the benchmark. And there's a huge gap.

I had some fear installing windows 11 (about loosing compatibility with some old Italian administration programs, mostly, but also with standard programs and related settings, login passwords etc.) but it was all ok and the upgrading process is a lot fast and full automatic till the end (less than 10 minutes). No need even to apply my windows services settings again after the upgrade: it was everything saved in the list as in my windows 10. As usual reading bullshits online reviews it is useful, but it can give negative impressions, without fondament. edited: my MB is an "old" MSI z490a pro and even with my modest i7 10700 vam VR with reverb g2 runs good. With windows 11.

 

[Jiraiya]

Do we have a benchmark on windows 10 -> windows 11 yet?
Would be good to compare a before/after benchmark to see how the upgrade effected things.

 

[keycode]

Do we have a benchmark on windows 10 -> windows 11 yet?
Would be good to compare a before/after benchmark to see how the upgrade effected things.

this can be more significant if you have a 12 generation intel, as they say there will be a specific software sinergy in windows 11. For the modest value my experience can have, besides the different stress with benchmark 2, as Mac G. told, I will not consider anymore windows 10. As I like more the new interface and I see no difference with performance, not only with vam, but even with the generic other software.... maybe, but it could be just my impression, it seems everything more fast. And yes, I was not a paranoid about having always a good clean system ... but my windows 10, before the upgrade, for sure had no crapware installed, besides steam vr, and world of warships, the only "game" I like to play, besides vam VR.

 

[keiseyku]

Has anyone else used an Oculus Quest 2 for Vam? When I used it on my i5-10400 RTX 3070 desktop computer I noticed it had very noticeable pre-render lag vs my Rift S having no lag. I looked at my previous benchmark posts and the two almost have the same parameters/resolution. Is there something I'm missing?

 

[JayJayWon]

Christmas present to myself arrived yesterday. Stupidly expensive system but pleased with the results:

To note, its on Windows 11, DDR5 RAM and came with CPU & GPU overclocking but not sure to what extent.

I just repeated the benchmark (on 10th Jan 22) on a completely clean VAM install. There's a significant boost:

Immediately repeated the original test on my bloated main VAM install and got the same result as before(i.e. ~175fps average overall)

 

[mostvanvege]

Here is an entry from Team Red. The little zen is overclocked (CO, PBO2), vga also got the MorePowerTool treatment so it can siph 300W for stable 2.6GHz++.
Ram got tweaked to IF. OS and VaM on nvme drive. I think its a resonable setup. Do not know if I miss something specific for vam, but based on the standard benchmarks the config gets where it should be.
VaM is a bit bloated so I may loose somewhere ~10%.

edit tips:
- leave OS on standard balanced power profile
- do not mess with SMT or other core affinities (I tought disabling extra threads is a good idea, but it lost me 30% on baseline 3 scene..)

 

[jakuzo]

I am representing mobile gamers here. Stock Acer Nitro 5. SSD drive. Tested after 2h of War Thunder at max settings. Firefox with only YT in the background. At full fans speed. CPU max 90 degress and GPU 65 degress. As you can see, I can recommend that machine. Great job with that benchmark. BTW. Thanks a lot for all the tools.

This Benchmark inspired me to borrow Quest 2. Here are results of running PCVR via AirLink on the cheapest 3080 laptop on the market. GPU sucked only around 90W. Biggest flaw of the machine. Temps and fans same as non VR. Nothing in the background.

 

[trety]

'Standardized' benchmark is a great idea, so finally decided to give it a shot.
The results... aren't so great:

Saw someone is using similiar machine in here so tried it on my testing vam install

Conclusion: No matter how much anyone can try to make an objective performance test, at the end of the day, it all depends on the ends user install.

First test on ~450gb vam install, nvme drive
Second vanillish, some basic stuff only installed, on HDD

 

[HolySchmidt]

That's remarkable. Seeing a difference of 20fps only by using a clean installation is impressive. Shows once again how bad VaM is optimized. Thank you for the comparison.
Would be interesting of what really makes the difference.

 

[JayJayWon]

That's remarkable. Seeing a difference of 20fps only by using a clean installation is impressive. Shows once again how bad VaM is optimized. Thank you for the comparison.
Would be interesting of what really makes the difference.

Yes that is interesting. I repeated my own benchmark on a clean install:

Benchmark Result Discussion

The 100% resolution per eye for the Vive Cosmos should be 1440 x 1700 pixels (2880 x 1700 pixels combined). Is that what your setting is? VR headsets usually run with slightly more than their native resolution. E.g. my own Oculus Rift CV1 runs at 2688x1600, although having only 2160×1200 actual...

hub.virtamate.com

Went from 175fps to 193fps

 

[trety]

Yes that is interesting. I repeated my own benchmark on a clean install:

Benchmark Result Discussion

The 100% resolution per eye for the Vive Cosmos should be 1440 x 1700 pixels (2880 x 1700 pixels combined). Is that what your setting is? VR headsets usually run with slightly more than their native resolution. E.g. my own Oculus Rift CV1 runs at 2688x1600, although having only 2160×1200 actual...

hub.virtamate.com

Went from 175fps to 193fps

Holy crap, i love that 12th gen performance xD
Over 50fps more in average\max and over 15 more in lowest 1%, at the most CPU heavy test comparing to my 10th gen >_<

 

[HolySchmidt]

Ok, so the real question is: how's behaviour in real game scenes, not using the benchmark. Which I found out right now

Setting: Simple scene (wooden floor, bed, 2 females, 3 lights)
Scene 1: no movement, just hanging in the air
Scene 2: one moving up and down (loop), other from left to right (loop) // only head is moving, all other atoms position and rotation is OFF so the full body is moving in the air

Nearly Vanilla installation
Scene 1: 186fps
Scene 2: 136fps

Totally overloaded installation
Scene 1: 156fps
Scene 2: 102fps



That's around 30fps less only for having "stuff" installed.
How "bizarre" is this?

@trety
Yes, 12th series is a big step forward for VaM 1.x

 

[trety]

Well, move out all urs custom morphs, and vars with morphs preload setted "on", then U will get these fps back.
The thing is... morphs are useful -_-
The only way is to keep them as much dupes free, and have all 'not morph packages' vars with preload setted up 'off'
30 fps is still not that bad. I'm already there too, guess will need to do some morphs cleanings soon...

I've been talking about it on discord a few months back. Was in much worse position back then, with my 'never cleaned morphs folders, with a lot of pre-var content'...
I was loosing to vanilla about 50fps in default 3p scene.

 

[MacGruber]

@HolySchmidt @JayJayWon @trety Could you try with MorphPreload disabled for all your packages? Probably make a backup of the folder where those package settings are stored so you can restore it later. I have a feeling this happens when you got LOTs preloaded morphs, although they are unused (and @trety's answer confirms). Should that help significantly, it would be possible to make a plugin that allows you to switch quickly between "performance mode" and "character edit" mode.

 

[TToby]

it would be possible to make a plugin that allows you to switch quickly between "performance mode" and "character edit" mode.

I would appreciate this very much! Great idea.
But will my custom figures still looking OK with this "performance mode"?
Figures with only Vanilla morphs are a bit boring, aren't they?
Do I need speed if I have to look at those lifeless Barbie dolls?

If preloaded but unused morphs are killing fps, then this is a big desaster.
A big morph collection is IMHO essential for everyone who is deep into doing look-alikes.
Due to my latest calculations, I have 10335 morphs installed.
I guess a bit overdone, but I am not alone with this high numbers.
According to MeshedVR some years ago, this shouldn't do anything to FPS, only killing the startup loading time?

To be honest, I had suspected the var files for the difference between vanilla and a stuffed installation.
First time I hear this about locally installed morph files.

 

[HolySchmidt]

Where can I turn preload morph on/off for all morphs? Only found the option in the package manager to deactivate preload morph for ONE morph each.

I have 8244 morphs in total.

 

[TToby]

Where can I turn preload morph on/off for all morphs? Only found the option in the package manager to deactivate preload morph for ONE morph each.

Yes, that is the way. One by one.

 

[HolySchmidt]

Ok, so then: good luck!