Development progress for Porydex has increased to ludicrous speed since my update post last month. I’ve finally made it out from under the mountain of backlogged work that had buried the project since late 2023, and now I’m even adding new features that I’ve been looking forward to adding for years!
The Big “Generations to Versions” Migration
It wasn’t easy. First, I had to spend days just rewriting huge swaths of code to be compatible with the database changes from half a year prior, when I started tracking lots of things on a per-version basis rather than a per-generation basis. (Technically a per-version-set basis, since most of the series comes in pairs, and the data is exactly the same between each member of a pair. Seriously, even when the games have version differences—like Pokédex entries!—each game has the data for BOTH games.)
Anyway, with those 2023 database updates, things like move descriptions are now version-set accurate instead of just taking the text from the last game in a generation and calling it the text for the entire generation. The “porydex.com/dex/rs/moves/pound” page will have the Ruby/Sapphire description of the move Pound, and the “porydex.com/dex/frlg/moves/pound” page will have the FireRed/LeafGreen description of Pound—which is a lot more verbose, for some reason.
Those text differences within a single generation are just a nice bonus though; the real goal was to be able to handle things like forms and species that are exclusive to games at the end of a generation. Like, all the ORAS-exclusive mega evolutions. You wouldn’t want those showing up on the list of Pokémon available in X/Y, right? Same with new moves, items, and so on.
Resolving the Icon Wars
Another thing I mentioned last month was the “icon wars” I got myself stuck in. Near the top of my Porydex To-Do List was a big bunch of folders of Pokémon icons from Scarlet/Violet, and from Legends: Arceus, and from BDSP, to be added to the website’s icon database.
It’s not difficult; it’s just tedious. Finding a sprite sheet splitting tool to split “sv-spritesheet.png” into “0001.png”, “0002.png”, all the way up to however high up it goes now. Then renaming all those files “bulbasaur.png” and “ivysaur.png” and so on because that just plain makes it easier to find specific icons. Then adding all those icons, for each game, into my big Pokémon icons Excel spreadsheet (because xlsx files are just easy to work with). And then using a LibreOffice command line command to convert that spreadsheet into a csv file, so MySQL can use a MySQL command to convert that csv file into a database table.
That’s the process I have after streamlining it for years. I store everything in Excel where it’s easy to modify, sort, filter, etc.—and then I run a few command line scripts to build the Porydex data files out of those spreadsheets. That’s how the entirety Porydex’s game data works. (Not the Showdown usage stats data, of course; that comes from Beyond.)
Anyway, the Pokémon icon stuff is done. As is a lot of the item icon stuff! I’ve integrated item icons from S/V and L:A. BDSP item icons are still on the To-Do List, sadly. As is reworking a lot of older icons, because the resource that Porydex originally depended on for item icons—the PokéSprite project—has ended with gen 8. That project split its item icons into categorized subfolders (like “berries/”), and thus so did Porydex, because I essentially copied their item icon folder wholesale.
But since my S/V and beyond item icon folders won’t be doing that, I want to go backward and un-categorize the old game icons for consistency. It’ll be another trivial but tedious task, especially since it’ll involve a “generation to version-set” migration for item icons. (There have been a handful of items that changed icons within a generation. Mostly things like the Bicycle or the Itemfinder. But, if I’m going for perfect accuracy, I may as well fix them.)
New feature! Ability flags
Finally, we’re getting to the good stuff.
New to Scarlet/Violet, abilities have a series of true/false data flags, mostly for how they interact with other abilities. Things like, “can it be traced by Trace?” and “can it be suppressed by Mold Breaker?” You know, boring technical stuff.
I love boring technical stuff.
I’ve added the ability flags to the “porydex.com/dex/sv/abilities/[ability-name]” pages, the same way you can find move flag data on the “porydex.com/dex/sv/moves/[move-name]” pages. Ever wonder which moves count as biting moves for the purpose of Strong Jaw? Or which moves activate Wind Rider? Porydex will tell you.
Still on the To-Do List: making it easy to search for abilities/moves via their flag data. Porydex’s inspiration, veekun.com, had an absolutely killer search page that lets you do things like that. I know I’ll get there someday.
New pages! Items
With the item icons for at least gens 8 and 9 squared away, I finally decided it was time to add items to the website proper. Now there’s a “/dex/sv/items” page that lists all items in the game*, and a “dex/sv/items/[item-name]” page for each individual item. The dex/item pages don’t serve too much of a purpose yet, for 99% of items. But, for evolution items (Fire Stone, Upgrade, etc.), its dex/item page will list which Pokémon evolve with that item in that game. I thought that’d be a fun feature, mainly to help people see how many Water Stones they’ll need for Pokédex purposes in a single game, for example.
New feature! Evolutions on the Pokémon pages
This took about two weeks, the first of which was me painstakingly going through the list of 50+ evolution methods that the games have now, and writing out code to write out text descriptions of them. I mean things like: “Level up, starting at level 16” “Level up, during the day, females only” “Trade while holding a King’s Rock” “Spin counterclockwise, for more than 5 seconds, during the day, while holding a Clover Berry.” I swear, that last one is NOT made up. Alcremie‘s evolution method took a day on its own, thanks to all its intricacies.
And then it took me the second week to figure out how to properly display a Pokémon’s evolution data on its dex page, aka how to convert an evolutionary tree into an HTML table that could support branching like Eevee or Gloom or Applin.
In the end, each step of the entire process is simple. But figuring out each simple step of the process was anything but. What to do about alternate cosmetic forms with different evolution trees, like Burmy? Okay, so the page for Mothim needs to go backward to its first form (Burmy), and then branch out to its alternate forms (Grass/Sand/Trash Cloak), and then find the evolutions for each of those root points.
What to do about Pokémon that have multiple ways of evolving into the same evolution, like Feebas? Okay, don’t turn that into separate branches like Tyrogue’s page does; batch the evolution methods together per evolution.
What to do about Gholdengo, which can be evolved into from two different Pokémon? (Roaming Gimmighoul and Chest Gimmighoul have different stats; they aren’t just cosmetic forms!) Cry. Cry is what you do. Or maybe just hardcode that tree to unbranch, since convergent evolution is apparently a thing now.
It’s not perfect yet. Gigantamax forms are currently in the system as “sibling” forms like the Burmy forms, but they should probably be displayed to the user as “evolutions.” Same with Mega forms. Those changes are near the top of my To-Do List now.
New data! Gen 2/3 descriptions for abilities/items/moves
My earlier example of Pound’s description in RS vs FRLG was a little misleading, because until a few days ago, I didn’t have gen 3 description data for anything at all.
For the last week or so, I’ve been going through text file renditions of gen 1-3 game decompilations, for the sake of finding all the ability/item/move descriptions, in EVERY language, and pulling them into Porydex.
Naturally, it’s a real pain in the butt, because games that old had very non-standard internationalization processes compared to today’s games. There’s a whole lot of manual work involved in this at every step.
To give you an idea of what these “decoded” game files look like, this is what I’ve been dealing with all week:
And to give you an idea of the manual process for, let’s say, getting the item data out from, let’s say, Gold/Silver versions:
- I open the 7 different translations of Gold in my text editor. 7, instead of 6, because Gold/Silver had an official Korean release! That wouldn’t happen again until Diamond/Pearl. (Fun fact!)
- Referring to my item data I already pulled from gen 3, (or gen 4 for Korean), in each file I CTRL+F for that language’s name for the item “BRIGHTPOWDER,” because that’s the item at index #3.
- Master Ball is at index #1, so why don’t I search for that? Because there’s a lot of other hits, like from the dialogue sections of the game. I chose BRIGHTPOWDER because that word is only in the entire file once.
- Highlight and scroll down until I get to the end of the item names list, with “HM07” (or “VM07”, or whatever it is in each language, but thankfully the “07” makes it clear every time).
- Copy into an Excel spreadsheet I’ve set up to hold all these names and descriptions.
- Did I copy these Japanese names right? Is that the full name for “Master Ball” or did I cut off a character? Cross reference with a Japanese Pokémon wiki to confirm
- Finding the item descriptions is trickier. In some games it’s right above the item names. In some, it’s far below. CTRL+F “The best BALL” to at least find it in English (that’s the first line of the description for Master Ball).
- Oh no, these item descriptions are split across multiple lines! Before I can copypaste these into my master spreadsheet, I need to copypaste them into a new tab in my text editor, and get each item’s description onto a single line.
- This is the most time consuming part. Getting this right in English, and then 5 languages I can decently guess at based on capitalization and cognates and punctuation across lines, and then 2 languages that are just symbols to me. A lot of cross referencing of Japanese and Korean wiki’s. And sometimes, YouTube videos. (The Korean name for Master Ball in the decoded Gold didn’t match what I had on file for Diamond, so I found a video of someone using a Master Ball in a Korean copy of Gold to see which name to go with. The Diamond name.)
- Once all THAT is done, now I have a spreadsheet with a tab of item names in every language, and a tab of item descriptions in every language. Now correlate their index numbers with the Porydex database’s item ids, so the THIRD tab in the spreadsheet, with its fancy INDEX MATCH MATCH formulas, can do its magic and turn this spreadsheet into the spreadsheet that can be converted into a csv.
- Porydex’s item ids correspond to the ingame index numbers in gen 4 and beyond. That’s why this part was necessary for gens 2 and 3.
- Make sure I didn’t miss numbering any items! Are there any “TERU-SAMA” items I accidentally gave an id? Are there any non-“TERU-SAMA” items I accidentally skipped over?
I haven’t actually done the item descriptions for gen 2 yet. Everything for gen 3, yes. Moves for gen 2, yes. Item names for gen 2, yes. But the GSC item descriptions are actually the last thing on my list for this general task. I’ll probably finish it in the next day or two.
Wrapping Up
And that brings me to today. That’s all the major stuff I did on Porydex since my last update. There was also plenty of minor stuff, like updating the code to run on PHP 8.3 instead of PHP 8.0, or a handful of data fixes mostly in evolutions. But this is pretty much how I spent the entirety of my last month.
At some point in doing all of that code fixing, all the dex pages for Scarlet/Violet finally started working. Hurray! I can finally, in theory, push all these code changes out to the public! The actual porydex.com website has been stuck in Sword/Shield for the last several years, but now that I’ve cleared the mountain, I can update the site and let people partake in the fruits of my labor. In theory.
Except, with all this rapid development recently, Porydex is extremely unstable on its foundational level (the database). As much as I’d love to update the site today, and have it start showing the last year of Showdown data from gen 9 battles, doing so would make it hard for me to make further updates to the foundational database tables (Pokémon, items, etc.), so I’m going to hold off a little longer.
But I don’t want to hold off for long. There’s no point in this project if it’s just for me. I want it to get back out there, more powerful than ever. I guess I should give myself a straight up deadline and list of minimum additional work before I push all these updates, so I can hold myself to it.
Anyway, what’s the next month going to hold for me? In addition to more Porydex development, next week I’m going to start playing Shining Pearl. I skipped BDSP when it first came out, but I crave for my ribbon master Pokémon (a shiny Smeargle from gen 6, named Escher because I’m a dork) to be complete once again.
