Establish an incentivized Beta software testing program to increase large-scale testing of new software on a significant number of PC rigs
I was prompted to submit this idea following the recent controller tracking debacle & this comment from an Oculus software engineer: https://www.reddit.com/r/oculus/comments/hobaop/foroculusriftsuserswhoare_experiencing/fxgw1sh/
I wrote a reply to that which I'll quote here:
The immense diversity of setups - different CPUs combined with different GPUs in different motherboards with different chipsets, power deliveries, USB ports, etc etc, with different RAM kits using different clock speeds & memory timings, interfacing differently with that exact motherboard at that motherboard's BIOS settings & that CPUs memory controller, with that version of Windows with those settings & those registry keys, running those background apps with those software versions & those settings... on and on and on it goes.
The factors involved in building the landscape for any piece of software to run in are luridcrously numerous. There's no amount of QC, QA, or testing that can account for all of them. As hardware & software gets more complex, which they are at a break-neck pace, testing is only going to get exponentially harder, and large-scale constructive community engagement with beta programs will become absolutely essential.
The best thing anyone in the community can do is report the issues. Oculus have a beta program, a log gatherer (IIRC in /Oculus install folder/support/oculus-diagnostics/), and a support channel. If you encounter an issue, mention it on public forums to contact other people who might have it, establish it's an issue, then en-masse use the log gatherer & send the info to Oculus Support. That's how you actually, constructively contribute to the fixing of the issue.
Given the increasing complexity & diversity of the home PC landscape, internal testing is getting more and more unfeasible & fallible. It's simply not reasonable or practical to set up an in-house testing area of thousands of PC rigs with tens of thousands of old & new components, different software versions, different OS settings, etc etc.
I suggest an incentivized Beta program, where users could receive something like store credits for participating in the Public Test Channel & submitting system logs using the Log Gatherer included in the Oculus software suite. Participate in the beta, provide feedback to Oculus, get store credits, buy free games. It would of course be up to Oculus to balance this in such a way that they gain significant beta program participation without seeing a significant loss of revenue, but I would argue that the loss of faith & goodwill from the community following issues like these are more harmful than some people getting a handful of free games.
You need a significant amount of users testing your software on a plethora of system configurations. In-house testing is no longer sufficient, and scaling that up to a point where it would be would instead be financially & logistically unpractical. This tracking debacle demonstrates that incredibly well, and it isn't the first time something like this has happened.
