I am planning on booking two separate flights, one from Helsinki to Bangkok BKK and then a separate flight from BKK to Phnom Penh. I would be bringing baggage.
Would I need to go through Immigration to get baggage and then go through check-in again, or would I be able to collect my baggage and continue to Phnom Penh without?
It depends.
You mention that you will be flying on to Phnom Penh on a separate ticket - this means that any automatic checking through of baggage is off the table, as your destination when you first check in is indeed Bangkok.
However, you can still ask at Helsinki checkin if your bags could be interlined through to Phnom Penh by the checkin staff, but whether this can be done entirely depends on the quality of the staff you speak to, the airlines you are flying with, and the agreements between those airlines.
If this is refused, then yes you will have to pass through immigration, then customs, then collect your baggage and checkin for your Phnom Penh flight as a brand new flight.
With such questions, even an old one like this (but still pretty valid), it is generally useful to mention the airlines involved. I have done such transfers, in BKK and other airports in Asia, a lot, and only had to pick up my luggage and check it in again once, as far as I can remember.
The main issue is whether the airlines involved interline, which is often the case in BKK with full-service airlines (LCCs on the other hand is a dead end). Even if they are not in the same alliance, they still might. I've done a lot of CX<>TG transfers (even with flights flown by Thai Smiles), on two tickets, with luggage, without any issue. HKG<>BKK<>RGN, HKG<>BKK<>VTE/LPQ, etc. Even once indeed HKG>BKK>PNH with CX and PG.
So if you're flying to PNH with either Thai or even Bangkok Airways, your luggage could be checked in all the way. In which case, when you arrive in BKK, you go back upstairs after security, and go to your next gate.
If the airlines involved don't interline, then you will have to pass through Immigration, pick up your luggage, pass through Customs, and go upstairs to check-in again.
As an aside, all international airports/terminals have luggage pick up BEFORE Customs (original question), since Customs' job is to inspect luggage, but AFTER Immigration, aka passport control.
There is at least one airport that has an arrangement to pick up luggage without passing through Immigration, in a specific situation: HKG.
When transferring from a plane landing in HKG to a ferry going to Macau or the Mainland, you go to a desk before Immigration, and hand over your luggage tag[s]. Staff will pick it up for you (they have PDA-like devices with a barcode scanner, loaded with the tags of the luggage they need to grab. They stand at the appropriate belts, scanning all tags until the device beeps), and it will be loaded on the next ferry.
But that's a pretty unique arrangement. It would be complicated to organize for air transfers, but could prove a worthwile venture...