JK2/JA Neural Upscaling

[Lancelot]

 

[Lancelot]

New upscale. This time, it has one major difference: It's in widescreen! The interesting part? No cropping was required.

The cutscenes were apparently all initially rendered in widescreen, then stretched to fit the 4:3 standard. The result was that some things looked a bit weird, such as planets looking egg-shaped. Before the upscale process, I simply converted the cutscenes back to widescreen.

 

[Acrobat]

Are these 4K textures worth it vs base textures?

https://www.nexusmods.com/starwarsjediknightjediacademy/mods/6

 

Or is the upscaling or AI upscaling just making the file bigger without adding anything or making it worth while I would assume?

[Lancelot]

7 hours ago, Acrobat said:

Are these 4K textures worth it vs base textures?

If you want bigger and sharper versions of the standard textures, this mod is for you. But if you don't see any significant changes from the standard textures in the screenshots, don't bother downloading the mod at all. If you decide to download the mod anyways, keep in mind that you need a strong PC in order to handle the textures properly without any significant FPS drop.

Quote

Or is the upscaling or AI upscaling just making the file bigger without adding anything or making it worth while I would assume?

Simple upscaling does pretty much nothing to images. You get bigger images, but no real details.

AI upscaling on the other hand is not only much more advanced, but also far superior to normal upscaling. Not only does it make images bigger and clearer, it also actually adds more details. And the results look extremely realistic.

I assume that the mod was made either with Gigapixel AI. Since 2019, when the mod was published, a lot more features have been added to the software, such as facial reconstruction. In fact, Gigapixel AI is already used by major studios to remaster the textures of their games.

[Botdra]

That's funny, I had just started on doing this myself for JA with Video Enhance AI. Your results seem a bit better than what I got, I assume it's because you're using Gigapixel and doing things frame by frame?

[Lancelot]

31 minutes ago, Botdra said:

That's funny, I had just started on doing this myself for JA with Video Enhance AI. Your results seem a bit better than what I got, I assume it's because you're using Gigapixel and doing things frame by frame?

I initially started with Gigapixel AI, but then gave up on that, as upscaling the videos frame by frame simply is too big of a task. When Video Enhanced came out, I moved to that software for that purpose.

I sourced some of the videos for the more recent upscales straight from the assets of the Gamecube version of Jedi Outcast. Like I said, the videos look far better than the PC cutscenes. The PC cutscenes are heavily compressed, while the Gamecube don't have that much of compression. That's why they are the perfect basis for upscaling. Jedi Academy on the other hand was trickier. The main problem was that some cutscenes were in 4:3 format, which resulted in planets looking like eggs. But once I converted them to the 16:9 format, they were much better material for upscaling.

The project was fun. In fact, I wanted to upscale all of the cutscenes, with the intention to get them into the game somehow. But since they added watermarks to the upscaled videos, I didn't continue the project.

But here's a question for those who have much better understanding with the engine: Is it possible to modify the game so that instead of ROQ files, it supports better files, like mp4 or avi?

[Circa]

I also have Video Enhance AI and thought about trying my hand at doing this. I've used for various other projects.

2 hours ago, Lancelot said:

But here's a question for those who have much better understanding with the engine: Is it possible to modify the game so that instead of ROQ files, it supports better files, like mp4 or avi?

I've asked this before and usually get the same answer: no. I forget exactly why, but I'm sure it has to do with encode/decode stuff. I feel like @mrwonko or @ent were people that I had asked in the past on Discord. Maybe they can give an answer as to why.

[Botdra]

4 hours ago, Lancelot said:

I initially started with Gigapixel AI, but then gave up on that, as upscaling the videos frame by frame simply is too big of a task. When Video Enhanced came out, I moved to that software for that purpose.

I sourced some of the videos for the more recent upscales straight from the assets of the Gamecube version of Jedi Outcast. Like I said, the videos look far better than the PC cutscenes. The PC cutscenes are heavily compressed, while the Gamecube don't have that much of compression. That's why they are the perfect basis for upscaling. Jedi Academy on the other hand was trickier. The main problem was that some cutscenes were in 4:3 format, which resulted in planets looking like eggs. But once I converted them to the 16:9 format, they were much better material for upscaling.

The project was fun. In fact, I wanted to upscale all of the cutscenes, with the intention to get them into the game somehow. But since they added watermarks to the upscaled videos, I didn't continue the project.

But here's a question for those who have much better understanding with the engine: Is it possible to modify the game so that instead of ROQ files, it supports better files, like mp4 or avi?

Hmm, I wonder if we could source the game files for the PS4 release of JA, it may have better quality video as well. I'd dump my copy but my PS4 is not hacked, unfortunately.

I've got an actual license for the upscaling software, so no watermark here. If you'd like to get in touch I'd be happy to use whatever settings you found luck with and export the finals so we can get them in-game. I've been working on increasing their FPS as well, so we could really get these looking much better than the originals.

[Lancelot]

@Botdra

Could you share your results? Screenshots are enough, so you don't need to upload an entire video.

[Botdra]

12 minutes ago, Lancelot said:

@Botdra

Could you share your results? Screenshots are enough, so you don't need to upload an entire video.

What did you want to see specifically? I don't have the same source files as you, and I'd be happy to use whatever settings you want to use. I just don't have the watermark issue so I'm happy to be the middle man so that we can get these to the public.

[Botdra]

Did a little dry run for you. This isn't quite optimized yet, but just a vague idea.

The JA cutscenes are rendered at a very odd aspect ratio, something vaguely similar to 7:6, requiring about a 30% horizontal stretch to properly render objects that are supposed to be a perfect circle.

The first photo is the original resolution, the second is what I consider the "best" workflow order (stretched by 30% first, then upscaled), and the third is what I consider overall inferior (I upscale the video, then stretch it by 30% to the correct aspect ratio), but it does have some perks. Stars look awful, very square and with more color bleed, but gradients like the one around the planet actually look a bit less block.

Next step would be doubling the FPS to 60 FPS, which is a pretty cut and dry process compared to upscaling.

https://jkhub.org/albums/C3OCt9N

[Lancelot]

5 hours ago, Botdra said:

Did a little dry run for you. This isn't quite optimized yet, but just a vague idea.

The JA cutscenes are rendered at a very odd aspect ratio, something vaguely similar to 7:6, requiring about a 30% horizontal stretch to properly render objects that are supposed to be a perfect circle.

The first photo is the original resolution, the second is what I consider the "best" workflow order (I upscale the video, then stretch it by 30% to the correct aspect ratio), and the third is what I consider overall inferior (stretched by 30% first, then upscaled), but it does have some perks. Stars look awful, very square and with more color bleed, but gradients like the one around the planet actually look a bit less block.

Next step would be doubling the FPS to 60 FPS, which is a pretty cut and dry process compared to upscaling.

https://jkhub.org/albums/C3OCt9N

Looks very good. The second image, with the fixed aspect ratio first and the upscale later, is definitely better. That was the procedure I did. Personally, I also added film grain. The right amount can mask some flaws of the upscales that are hard to avoid.

Higher FPS are definitely not a bad idea. After all, most players are able to play the game with 60 FPS (at the very least), so it would definitely add some consistency.

[Botdra]

6 hours ago, Lancelot said:

Looks very good. The second image, with the fixed aspect ratio first and the upscale later, is definitely better. That was the procedure I did. Personally, I also added film grain. The right amount can mask some flaws of the upscales that are hard to avoid.

Higher FPS are definitely not a bad idea. After all, most players are able to play the game with 60 FPS (at the very least), so it would definitely add some consistency.

Film grain is a good idea, adds a bit of perceived sharpness as well.

If you're interested, you could send over the source files along with any settings or film grain overlays you'd like me to use (I've also got several film grain emulation plugins etc. if need be) and I could process everything and send it back to you this weekend.

[Lancelot]

@Botdra

You already have the files you need, so I don't think that there is anything I could send to you.

[Botdra]

7 hours ago, Lancelot said:

@Botdra

You already have the files you need, so I don't think that there is anything I could send to you.

I meant the Gamecube JKII files, if you wanted to do those as well. Otherwise, I can definitely handle the JA stuff.

[mrwonko]

On 6/30/2022 at 11:13 PM, Circa said:

I've asked this before and usually get the same answer: no. I forget exactly why, but I'm sure it has to do with encode/decode stuff. I feel like @mrwonko or @ent were people that I had asked in the past on Discord. Maybe they can give an answer as to why.

In principle, adding support for new video formats should very much be possible, but nobody has done it yet, to my knowledge.

[Lancelot]

5 hours ago, mrwonko said:

In principle, adding support for new video formats should very much be possible, but nobody has done it yet, to my knowledge.

If somebody could make that happen, that would be another milestone for the game. I mean, as far as I know, ROQ files can only handle a certain quality, which means that videos need to be heavily compressed in order for the game to actually accept them. Also, another issue is that the game doesn't support widescreen videos. That could be another thing to look out for.

[Botdra]

Just to update, I'm working on this, it just requires more time than you'd think.

Based on the tests I did above, you'll see that the version where I correct the aspect ratio and then upscale looks great, but also ruins stars. They look horrendous. Square, tons of color bleed, just gross. The version where I upscale and then correct the aspect ratio stars look great, but everything else looks more pixelated and has more artifacts.

So naturally, if you're gonna do something, do it right. I'm making both version of the files, and manually masking out the areas I want to keep from each set of files based on the scene. Only then will I convert them to 60fps. It's time consuming, but this will result in the best quality final product. Results so far are promising.

[Botdra]

 

Alright, everything is done. The only tricky part is that I can't seem to get them in-game as Quake Video Maker is crashing on me, so I may have to pass these files off to someone else for that step. Everything has been upscaled 4x, with the aspect ratio corrected, then converted to 60fps. The resulting resolution is 2664x2048. I'll probably add a touch of film grain at some point as well, but this looks pretty good for now.

[Botdra]

Conversion process is giving me a bit of a headache. Instead of wasting time I'll just drop the MP4 versions here for anyone who wants to convert them and get these in-game.

https://www.mediafire.com/file/1wbov77z5gjjsev/UpscaledJACutscenes.zip/file

[Lancelot]

 

2 hours ago, Botdra said:

Conversion process is giving me a bit of a headache. Instead of wasting time I'll just drop the MP4 versions here for anyone who wants to convert them and get these in-game.

https://www.mediafire.com/file/1wbov77z5gjjsev/UpscaledJACutscenes.zip/file

I moved this topic to the WIP section. Even though it was me who started it, feel free to post whatever you have here. It's great that you've made these videos avaliable for everyone.

Anyone who can add something to this project are also welcomed here. I think it's a project that can be achieved through teamwork.

[mjt]

Looking at how some games came along with bink as video format - it seems that some of the idtech4 source ports for doom3 / quake4 have implemented ffmpeg into the engine to support different video formats and more high resolution content. In principle you'd have to find someone willing to do the same here. Or find a different solution. I think .roq won't hold any of the upsampled data. It's just too limited.

[Lancelot]

3 hours ago, mjt said:

I think .roq won't hold any of the upsampled data. It's just too limited.

Exactly. ROQ videos are simply too obsolete to handle HD data. That's why better alternatives are required to maintain the quality.

[Scoundrels Cantina]

On 7/11/2022 at 7:02 PM, Botdra said:

Conversion process is giving me a bit of a headache. Instead of wasting time I'll just drop the MP4 versions here for anyone who wants to convert them and get these in-game.

https://www.mediafire.com/file/1wbov77z5gjjsev/UpscaledJACutscenes.zip/file

Hey man I just wanted to ask you if you've upscaled the Jedi Outcast Cinematics by any chance? I've been searching for those for 2 years now so that I could make some Jedi Outcast and Academy based lore videos in the best possible quality I can. I can't settle for the old crappy quality XD