You can't burn them with the kerosene since they don't stay down long enough (learned this the hard way). You have to blow off their head. This isn't the easiest thing to do so you have to decide if they are worth killing or if you can avoid that area/run away from them.

Later in the game there will be another way to take them down.

I did burn one crimson head, I just had to shoot it down several times, before it stayed down long enough to be burnt. I was just curious if there was an easier way, as I have one in a place I need to get through that ripped me up last night right before I quit.

is that eventually you'll find a grenade launcher and Incendiary shells. These shells explode and fry zombies, which is pretty helpful because it burns up the crimsons once and for all. Unfortunately there's only 6 shots so use them wisely. I think i got the shells from barry after meeting up with him after awhile. That way you don't have to carry around both the kerosene and the lighter. Or you can just run and kill the crimson, light them up, and run back to the save point and switch your weapons. Something I should have done but messed up was to secure the save room and item box, I had 2 crimsons outside the one in the top left corner of the map outside the save room, and as you can imagine it was a frantic and tough time getting in and out of there.