hwd45

Members
  • Content Count

    574
  • Joined

  • Last visited

  • Days Won

    32

hwd45 last won the day on September 10

hwd45 had the most liked content!

Community Reputation

269 Excellent

About hwd45

  • Rank

  • Birthday 12/11/1995

Profile Information

  • Gender
    Male
  • Location
    Abingdon, Oxfordshire, England, United Kingdom

My Tamagotchis

  • My Collection
    V1 x 3
    V2
    V3 x 2
    V4
    V4.5
    V5
    Music Star
    Tama-Go
    Entama
    Eevee × Tamagotchi
    Tamagotchi On
  • Favorite Tamagotchi
    Version 3
  • Favorite Tamagotchi Character
    Mametchi

Contact Methods

  • Website URL
    https://youtube.com/hwd405

Recent Profile Visitors

787 profile views
  1. Although it can be difficult for a human to get their head around, the operations used in decoding a password are actually really simple for a computer written in an assembly language (like the one Tamagotchis are written in) to perform. In some senses it's kind of the simplest way they could do it while still maintaining some level of username encryption!
  2. Interesting! Either I've forgotten to add a BIN2DEC somewhere in the generator or there's something I've totally not accounted for.
  3. Huh. And Byte_5 was 0 when you changed Byte_4? That's interesting because it potentially shows some addition a structure I wasn't aware of.
  4. d_item_user can be anywhere from 0 to 255 unfortunately. It's a lot of trial and error at the moment, and I've still not been successful in finding any semblance of a pattern in these numbers.
  5. Unfortunately due to the nature of the generators in that topic, they stopped working when the Tamatown servers went down in 2013, and there's no way to fix them. Until someone creates a working generator, it's impossible to generate new passwords for the V3 (give me a couple weeks and maybe that'll change ). I don't think that's really relevant to this topic, did you mean to post somewhere else?
  6. Hopefully these will answer your queries: How did you figure out where the letters are stored by connecting japanese and english devices? Maybe I didn't explain well enough in the previous post - basically, I worked under the assumption that the index that's used matches the order the characters appear on Japanese devices, and the English characters are placed in such a way to fit around that. When you connect a Japanese device and an English device, sometimes the name in the friends list will appear incorrect on the other device. Hiragana are present on both devices and so they always show correctly, but most of the punctuation symbols are replaced with letters on the English devices. Connecting Japanese and English devices together and seeing how the names change reveal the correspondence between letters and symbols - ー is replaced by A, ~ is replaced by B, … is replaced by C and so on up to .being replaced by J. Then, on the Japanese devices, you have ! and ? which don't change during the connection process. Afterwards you have & which is replaced by K and so on. You said that the alphabet replaced punctation starting at 81 (A), but why is A assigned to the number 1? I'm not sure, but I assume it was convenience on the part of the programmer. Adding 80 to every value just results in a sort of redundant couple of bits, and it means that 7 bits have to be used to store each character instead of 5. Choosing the 5-bit option means they can get the most use out of the username - using all 7 bits could result in them being unable to use the same method to produce a username constant which actually uses all five of the characters in the username. Where do the values 70 and 199 come from? I'm not sure! They were basically what just popped out of my calculations, but they work for all usernames and items so I guess they were just chosen to add some additional "encryption". It also means that they could use the same password algorithm on later versions and all they'd need to do is change these two numbers to guarantee that passwords that work on the V3 won't work on, say, the V4 (I'm expecting to tackle this version's passwords next so this'll be the first thing I try). Is there any way we can help by generating and testing passwords? Yes! Though it will be somewhat time consuming, so if you'd like to give it a go I'd recommend doing something else at the same time so you don't get bored out of your mind. Let's recall what the two unknowns are - the item-user variable d and the fluctuation variable E: Calculating d One thing I'd like help with is calculating some values of d for different items and usernames. Here's the process: Copy the generator spreadsheet Set Byte 5 to zero Set Byte 4 to zero Choose a working ID number (there's an item ID list in the spreadsheet) and set the ID field to that number Cycle through different vaules of d_item_user to generate different passwords, trying each one until one works If none work then either an invalid ID has been chosen or the spreadsheet doesn't work as intended. Either way, it'd be good to inform me of the issue! Eventually you should find a value of d that works. Note down the value of d somewhere along with the item ID and username and try a few different values of Byte 4 to check that it always works. If it doesn't, inform me of this too! Assuming that it always works, this value of d will either be of opposite parity to the item ID (i.e. one is odd and one is even) or the same parity (both odd or both even). All values of d will follow this rule for your username, so you can limit your search for other values of d for other item IDs in the same manner. For example, if my item ID is 128 and the value of d associated with it is 78, I know that for item ID 129 I only need to try values of d that are odd. Calculating d E is related to the values of the ID, the username and the value of Byte 4. Here's how to calculate it: First, set Byte 4 to zero and Byte 5 to one. Try using the same item ID and the value of d that was calculated in the previous part and check that the output password works If it does work, change Byte 4 to a few different values to check that it always works. If it does then note that down. If it doesn't work, or if it only works for some values of Byte 4, then duplicate the generator sheet and in the copied version edit the "E" and "e" values on the right to 0, as well as Byte 4. This means deleting the formulae that are in those cells, even if they currently appear to equal zero! Now change the value of d by adding or taking a way an odd number and trying the result until a password works. You will usually only need to add or take away a number less than or equal to 15, so try these first. What I had written here before is not a good idea. Instead, keep d the same as it was before - and definitely set e to zero - but edit E to be one of the following integers until a password works: 1, 3, 7, 15, 31, 63, 127, 255 Jot down the working d E value, change Byte 4 to something else (increment it by 1, for example) and try to calculate a working value of d E again. Keep a record of all the working values of d E, the value of Byte 4 that they correspond to, the item ID and the username. If this all sounds a bit much then no worries! It's what I'll be trying myself, but of course I can only try it on one username currently.
  7. I've made sufficient progress that I think it's time for an update - let's once more go over how the passwords are generated. Passwords can be thought of as a five-byte number. Using zero as an example we have: 00000000 00000000 00000000 00000000 00000000 Each byte does something different. The rightmost two bytes seem to be the simplest to understand - they're essentially just random numbers that determine the rest of the password. The leftmost byte (which we will call B_5) may only be 0 or 1, while the other byte (B_4) can be any number from 0 to 255 (00000000 to 11111111 in binary). The other three bytes are in part determined by the username. Let's consider an example username: "TMGC!". Each letter of this username can be represented by a five-bit integer corresponding to its place in the alphabet... except, it's not quite that easy. If you've ever connected a Japanese device to an English one before, you'll know that they don't always handle eachothers' character sets perfectly. Here's a summary of the character set used by the Japanese devices: Characters 0 - 80: Hiragana Characters 81-90: Punctuation Characters 91-92: ! and ? Characters 93-105: Punctuation and symbols I distinguished ! and ? here for a reason - they're also in the English character set. The English characters - as they appear on the device - cycle through each of the letters before cycling through the symbols. However, connecting an English device to a Japanese device hints that, internally, the ! and ? characters may be stored in the middle of the alphabet instead of at the end. You see, the alphabet replaces the punctuation from the Japanese devices. All the hiragana characters remain in English devices, but from character 81 onward the letters begin replacing the symbols. Characters 81-90 get replaced with A through to J, but that's when something different happens. The ! and ? characters on the Japanese devices are actually the same as the ones on the English devices, so connections between devices with these characters in the name will result in them remaining when viewing the names in the friends list. Then, characters 93-105 get replaced by K through to W. The Japanese devices have no characters for 106-108 so X, Y and Z in an English name will appear blank on a Japanese device. What this all means is that the naive approach of assigning A to 1, B to 2 and so on is not necessarily the right approach. Instead, we should assign numbers to each of the letters as follows: Blank character - 0 A - J: 1 - 10 !, ? - 11, 12 K - Z - 13 - 28 Converting each of these numbers into a 5-bit binary integer and concatenating them in a certain way gives us our username constant - returning to our example of TMGC!: T = 22, 10110 in binary M = 15, 01111 G = 7, 00111 C = 3, 00011 ! = 11, 01011 We concatenate them in reverse order: 01011 00011 00111 01111 10110 Splitting this up into bytes: 0 10110001 10011101 11110110 As I understand, this constant is probably XOR'd with the password at some point in the encoding / decoding process. However, I'm not really very confident in what the leftmost 9 bits do, so we're going to ignore them for now (they probably still play a predictable role in the passwords though). With the two bytes we have left, let's XOR the first with 70 and the second with 199. This gives us: 11011011 00110001 In decimal, these are 219 and 49 respectively. You may recognise the 49 as being the C_user constant for TMGC! that I mentioned in an earlier post. XORing 219 and 49 gives the D_user constant I also mentioned before. Next, we XOR the left byte by the ID of the item we're producing a password for, and after that we XOR both of these bytes by 106*B_5 - if B_5 is zero this has no effect on the password. The two bytes we have left after all this are the rightmost bytes of the password, B_2 and B_1. So, using item ID 128 as an example and setting the left bytes to zero we've been able to produce the following password: 00000000 00000000 ???????? 01011011 00110001 (If you're wondering what happened to B_2, that's the result of XORing with the item ID 128). The only bit left to figure out is the middle byte, but sadly this is where it all gets a bit unclear. Here's what is known about B_3: When B_5 = 0, B_3 XOR B_4 is a variable which depends only on the username and item ID - changing either of these will change the value of B_3 XOR B_4, but equivalent passwords will always give the same result of B_3 XOR B_4. When B_5 = 1, B_3 XOR B_4 fluctuates slightly and these fluctuations depend on B_4 and the item ID. B_3 XOR B_4 will either be: Always the same parity as the item ID Always the opposite parity to the item ID I haven't really figured out what's going on with Byte 3 yet but this information means that, at the very least, we can narrow down the number of potentially valid passwords from the 10,000,000,000 figure we had prior to finding any of this out to only 128 potential passwords. As it turns out, for the above username / item ID / B_5 trio, B_3 XOR B_4 = 78. Since B_4 is zero here, B_3 is 78: 00000000 00000000 01001110 01011011 00110001 After B_3 is generated, the entire string is converted into a decimal integer - in this case, it's 5135153. This is padded with zeroes to make a ten digit integer - 0005135153 - and then the first five digits and the last five digits are switched to give the following: 35153 00051 I've tested this password and can confirm that it works So that's (most of) how the passwords work! Next I'm going to brute force a bunch of passwords to see how the B_3 XOR B_4 value changes. It nearly correlates with the item ID (XORing with the ID results in something which generally remains constant) but after a while it starts acting completely unpredictably, so more research is needed. Once this one variable is understood, the V3 password system will be entirely cracked (apart from travel, king and parent related passwords because there's not enough information to crack those). I've produced a sort-of-working password generator for people to copy and use (I think you can copy this entire document to your own google drive but I'm not certain?) but it still requires finding the value of a mystery constant and it generally doesn't work when B_5 = 1. If you're willing to brute force up to 128 different passwords to get what you want, then this spreadsheet should help you do that. I can't guarantee that I haven't made a mistake in the formula somewhere though!
  8. As far as I know debug mode doesn't really provide any additional tools for checking the password system. You'd think this would be a pretty important thing to debug though! Right now I've hit a bit of a wall in the research - I've figured out a few more bits of information that narrow down some of the constants a bit further but it's not really enough to use as of yet. A single working password for a given username is now enough to narrow down the number of password possibilities for each item to 128, which is obviously a lot better than the 10 billion we started with. I'm gonna try and experiment with my device to get a password for every item ID for my username and see if that helps me pin down the algorithm any further. Naturally, though, since you have to retype the entire password if it's wrong and there's a few seconds of delay due to there being an "ERROR" screen, trying lots of passwords can be kind of time consuming. Part of me wonders if the debug mode also has the same way of handling errors or if the "no" sound would play similar to the Music Star's item passwords. I guess if anything would help debug the password system, it would be to make it nicer to use. Thinking about it, there's two different firmware versions of the V3 in America, too. Since you rarely had to type in passwords from the device into Tamatown, I doubt the password system has any versioning aspect which alters the way it works... though I do wonder if perhaps the earlier version had a glitch in the decoding algorithm or something that they needed to fix. --- Since I've made so much progress on these passwords I'd like to take a crack at other passwords too. V5 and the Japanese releases seem to be well-understood at this point, so there's a few other passwords I'd like to attempt to understand instead - V3 King passwords, V3 Travel passwords, V3 Parent and Grandparent passwords, V4 login and logout codes and all its other passwords and the Music Star item codes. I have a feeling most of these are going to be a lot more difficult to crack but they might be fun to attempt if I can't make any more progress on V3 passwords
  9. Previously I mentioned that I don't think we're close to cracking anything. Less than a day later... I've changed my mind I'm still playing around with things, but I've noticed some really solid patterns beginning to emerge. Let's establish some variables to begin with: we begin with a password consisting of two 5-decimal strings, (P_1, P_2). By concatenating P_2||P_1 we receive a 10-digit decimal number P. Now convert P into a five-byte binary number, and we'll call these bytes B_1 through to B_5 with B_5 being the leftmost byte. Let's use an example: 70545 42729 is a Passport password for the username "TMGC!". This makes up our (P_1, P_2). Next, we produce P by concatenating: P = 4272970545. In binary, this decimal is represented as 11111110101100000101101100110001. We'll add a few leading zeros to make it 40 digits long: 0000000011111110101100000101101100110001. This gives us our five bytes: B_5 = 00000000 B_4 = 11111110 B_3 = 10110000 B_2 = 01011011 B_1 = 00110001 As mentioned last time, applying the modulo 256 operator to the password gives us a pair of numbers which we called A and B. This is essentially equivalent to the value of B_1 - it only ever takes on two possible values for a given username. As it turns out, the value of one can be obtained from the other by applying the XOR 106 operator to it - this is true of all usernames! Hence, we have a constant C_user which relates to the username, but I'm not sure in what way it relates just yet. But some things have been observed: If B_5 = 0, B_1 = C_user If B_5 = 1, B_1 = C_user XOR 106 I'm not sure what happens if B_5 is 2, but I'm willing to guess it just loops back round to being C_user again. If I had to guess, I'd say B_5 can only take on the values of 0 or 1 and it takes them on randomly. For user="TMGC!", C_user = 49. So that's two bytes down, but it depends on a constant C_user which we're not currently sure how to calculate without obtaining passwords first. Now let's look at B_2: B_2 = ID XOR B_1 XOR D_user There's two new variables here. ID is the item ID variable - for example, Passport is 128. These IDs can be found in some of the archived Tamatown XML files. D_user is another variable that relates to the username in a way I've not quite figured out yet. I'm not sure what B_3 or B_4 do just yet, or if they even matter! But given how so far this seems to really strongly depend on the byte structure I'm willing to guess they've got their own roles too. In other words... I'm so close, I can almost taste it. I think at this point I just need lots of username and password data to calculate lots of values of C_user and D_user, so any information would be strongly appreciated! One thing that should be said about all this is that it's all slightly wrong for passwords that depend on a parent or grandparent character or travel tickets. These ones seem to be character independent but it doesn't seem to change the algorithm enough to make it unusable; but without data about what character people were using when they used the travel ticket and so on there's not much I can do to crack them. There's plenty of hope for the rest of the items though! Until next time!
  10. So after some fiddling around with passwords I've decided... it's time for another update. After looking over the old password generators and the information that was sent to the servers using the .cgi file upon login, I noticed that the login password, which is seen on the device in the form: ABCDEFG HIJKLMN Is formatted in the login information as: HIJKLMNABCDEFG I thought about this for a moment and noticed that, this entire time, I've been working with passwords assuming the top row to be the first five digits of a ten-digit number. What happens if I swap the two rows? In other words, what happens if I treat a password like "01234 56789" as the number "5678901234"? I went back to my large list of Passport passwords for the username "TMGC!", reordered them as above, arranged the rearranged passwords into numerical order, found the differences between consecutive passwords and arranged these differences into a list in numerical order. When I did this before - with the top row as the first five digits - I found 92 unique differences between passwords with some differences appearing multiple times. When I did this with the reversed passwords? There was only 37 unique differences. Not only that, but certain differences appeared significantly more frequently than others - I started with 218 passwords, generated 217 differences and of those 217 differences 100 were one of three specific differences. In essence, there's a hell of a lot more structure here than there was when we used passwords in the more intuitive way. Not only that, but the awkward multiples of 329 and 589 that we had before entirely disappear here. Instead of being slightly awkward sums of slightly awkward numbers, this time all the differences are whole multiples of 65536 = 2^16. Awesome! Okay, so there's one small catch. One of the differences wasn't a perfect multiple of 65536. In fact, it's not even close, being off by over 10,000. So what happened here? Was there a typo? Well, the thing is, if there was a typo, we'd expect to see the next difference be off by the same amount as the previous one, but it's not. It's alone in being totally, totally wrong. This will come up again later, as you'll see. As a result, if we instead take each of the edited passwords modulo 65536, we find that the first half are 23345, and the second half (appearing after the one difference which wasn't equal to 0 mod 65536) are 12635. This is a level of structure that's never been seen in the V3 passwords before! This also explains why all the passwords have digits of the same parity in position 5 - after rearranging, this digit becomes the final digit of the number, and because the difference between any two passwords is an even number, either all passwords are odd or all passwords are even, and hence the same is true of the final digit. I realised that looking into the values of these rearranged passwords modulo 65536 doesn't require having lots of passwords for the same item and username, so I went back to my database of passwords from other usernames and for other items and performed the same operations on those passwords. Passwords corresponding to the same username and item pair showed the same thing again - after using the modulo 65536 operator, they would always come out with one of two numbers. As a few examples: TMGC!, Passport - 12635, 23345 ICHI, Passport - 3268, 26286 TINK, Shoes - 11240, 16770 When I did the same thing to different items for the same username, the pattern wasn't immediately obvious, as every item had its own number (presumably, every item had its own pair of numbers as before, but I wasn't able to confirm this). So, I did what I always do - copied them, arranged them into numerical order and found their differences. I was surprised to find that, once again, the differences are almost always a power of two (2^8 this time), with some minor exceptions, resulting in two different values coming out as a result (I actually didn't need to arrange them into another list and perform the operator on them too - I could've just taken each of the modified passwords mod 2^16 and I would've got the same result). One value corresponded to lower password numbers, the other corresponding to higher numbers. So basically, this is what we know up to this point: All passwords for a given username, after rearranging the top and bottom rows, take on the values A and B upon applying the mod 256 operator. The values of A and B depend on the username, so different usernames may result in different values for these constants. For each username and each item, there are a pair of numbers C_item and D_item such that C_item = A mod 256 and D_item = B mod 256 and, for any (rearranged) password for this item and username pair, either P = C_item mod 65536, or P = D_item mod 65536. There seems to be a point Q such that, for P < Q, P = C_item mod 65536; and for P > Q, P = D_item mod 65536 (perhaps after switching the definitions of C and D). Each different item has a different pair of C_item and D_item. By the looks of it, the information from C_item and D_item is used to not only determine what item to give the player, but also to check that the password is compatible with the username. This is information that might allow us to predict passwords given passwords we already know, but generation is still a way off. Basically, this just proves there's a lot more structure than we thought there was! Now let's look at a few of the values of A and B for a few different usernames: TMGC! - 49, 91 TINK - 155, 241 ICHI - 174, 196 MAUDE - 130, 232 GRACE - 42, 64 TAMA - 155, 241 WHALE - 180, 222 SANTA - 152, 242 RAPHA - 153, 243 Eagle-eyed readers may have noticed that TINK and TAMA have the same numbers. Perhaps that's related to the fact that the first and last characters are the same in both (the last character being a space)? There's also something going on with RAPHA and SANTA, perhaps it's the As? Or, if we look at only the first and last characters again, we might notice that R and S also only differ by one. More passwords and usernames are needed! If you're wondering what happens to them at greater powers of 2, it becomes less interesting - some of the values we were looking at differed by exactly 256, so once we start looking at 512 instead, more values start to emerge. If we look at modulo 64 instead, the values from TMGC! also match those of TINK and TAMA - maybe that's a T-thing? So yeah, basically... we're getting somewhere, but I still don't know if we're anywhere near to cracking anything. Only time will tell!
  11. I didn't forget, I was entirely unaware! Hahaha It makes sense though, given the V5.5 is version 36.0 in all regions. I guess the character-related integers in the password are to do with the character's ID and gender? Looking at the image above, a male futabatchi would have ID 05 and Mataritchi would have ID 11.
  12. This is all incredible work! Also, can confirm that there's (probably) only four regional releases - they seem to correspond with the region codes: 0 - Japan - ROM versions 28.2, 28.3 1 - Asia / Oceania - ROM version 30.1 2 - US - ROM version 32.1 3 - Europe - ROM version 34.1 Not sure if there's any difference in the passwords between the two Japanese revisions and the ROM version of the Spanish release is currently unconfirmed (but it's probably just 32.2).
  13. This is really impressive! I was aware of the fact that a lot of the V5 swfs were archived but without being able to get past the initial loading screen I had no idea it was actually this complete and functional. I also really appreciate that you've collected the files in a google drive for people to easily access! It's important to find ways of preserving these files in places that are easily accessible.
  14. Recovering Tamatown is a popular topic, so let's go over all the most popular queries in one: Preserved Tamatown Files I've got a spreadsheet here containing all that's currently known about what files have been preserved and which are missing. So far, the Wayback Machine has been our only source for recovering the lost Tamatown files, and no other attempts at recovery have been successful. You may have noticed that there have been several attempts to "revive" Tamatown by stitching together the few files we have. These are often sensationalised or misunderstood, so let me explain with a sort of stock reply I give to people first hearing about the revivals: Password Generation So, if the Tamatown files weren't fully preserved, one might wonder if the elements that allowed for password generation were? There were password generators that were active in the past, after all. However, as you correctly speculated, the generation was done server-side: A .cgi common gateway interface file was used to access server-side password generation programs and so by their very nature these server-side files are not cached or archived as only the .cgi file is directly accessed and these files are simply information request files that gather information from the servers. All previously accessible password generators simply took in input parameters like item ID or user name and they'd perform the same server request that is done when accessing Tamatown normally. As such, when the servers went down, so did the generators. Password Cracking I made a thread a few months ago here about the potential for cracking passwords with brute force. The tl;dr of it is that while it's potentially possible to further our understanding of the password system this way, it would take immense amounts of effort. I've been able to figure out something of a pattern in the Passport passwords for the username "TMGC!", but I could only do so because I was able to obtain a list of several hundred passwords which all gave the same outcome. If I had access to a similarly large list of passwords for a different item or username then I'd probably be able to make significant progress on understanding the passwords, but alas, as far as I can tell no such list has ever been produced. This means that, unless a miracle happens, there's only one way to crack the passwords: ROM Dumping Of the connection series devices, only the ROMs of the Friends (specifically ROM version 8.0 00 USA) and the Tama-Go (ROM version 7.0 01 32.0 USA) have been dumped so far. It's not impossible to extend this to other devices too - there are plenty of people capable of doing this - but a lot of efforts at the moment are focused towards newer devices, of which multiple have been successfully dumped. Furthermore, there's also the issue that the V3 would have likely used a SunPlus microcontroller instead of a GeneralPlus one like the Tama-Go uses - as such, it's harder to find the required data sheets to understand and successfully dump the device's ROM. Basically, a ROM dump will definitely happen at some point, but I don't really have any idea of when that'll be, sadly. I've got my fingers crossed, though.
  15. A couple pages back I posted a link to a document I've been working on where I'd listed all the known Tamagotchi firmware versions. Previously, the document had basically just copied the version list post I made, but since this turned out looking like a bit of a mess, I've completely remade this part of the document to make everything much nicer to look at. This list is a bit too big to put into a single Tamatalk post now (without making the page really long!) but it now also includes all the known gaps where extra information is needed. You can find it here. Any feedback would be appreciated!