Include the rear derailleur pulleys in your drivetrain inspection. Remove accumulated grime with a brush and a suitable degreaser. Apply chain lubricant to the chain rollers; coating the pulley wheels attracts more dirt. After maintenance, lift the rear wheel and check shifting across the cassette.
If removal is needed, photograph the arrangement first. Upper and lower pulleys can differ, so service one at a time. Preserve the order and orientation of covers, seals and bushings. Construction varies: some pulleys use bushings, others use bearings.
For assembly, consult the manual for your derailleur and its bolt torque specifications. Where rotation arrows are present, they must match the direction the pulleys turn when pedalling forwards. Substantial play accompanied by noise while riding warrants a workshop assessment for pulley replacement.