SJ Kingpin Shims on my 2008 Jimny?

I had an accident a few years back that smashed the passenger side front, only some cosmetic damage and my steering was affected too.

I serviced the swivel join on that side (Replaced both swivel seals, kingpins, bearings + races, oil seal, and etc) and decided I could not be arsed to do the drivers side. When reassembling this side I also included these shims albeit blue (see below) that were present as I believed them to be part of the joint. However I have discovered these are for an SJ?

Since then the passenger side swivel joint was leaking oil (which I suspect was caused by the inner oil seal failing) so I serviced that one again, and then decided I should finally get around to doing the driver's side on this occasion.

Whilst disassembling the driver's side I noticed there were some shims also like the ones pictured above, I have not refitted these shims.
 
So when I replaced the kingpins, I did not replace them. Since then my Jimny has a touch of death wobble when around 45-50mph, and while braking.

So my questions are, what were they doing there in the first place, and  should I refit them or just use the standard Kingpin Shims?

Many thanks!

They are not the correct part for a Jimny. Jimny kingpins should be fitted directlty to the swivel with just a thing layer of sealant. To shim them you add round washer-type shims that go on the round shoulder of the kingpins where the bearing inner race sits.
Using those SJ shims on a Jimny will almost certainly lead to inadequate bearing preload, which will result in some steering wobble, movement in the kingpins and wear on the driveshaft seal causing the diff oil to leak out.