Pick it up and come back to the altar to slot it in! You will find the blue dial (pictured above) in a cabinet inside the alcove just right of the entrance to the Church. On interacting with the lever on the wall on the left side of the puzzle, the altar opens up and reveals two dials – one green and one red.
Here is the step-by-step on how to solve the puzzle on the stained glass inside the Church:
RE4 Remake: Church Window Puzzle Solution