Green concrete really doesn't have much strength until you get some weight on top of it i.e --building.  You can back fill now, just don't pack too close to the wall.  However, if you are going to put down gravel inside, then you can put gravel next to the wall and run vibrator over the whole thing before you pour concrete floor.