The Architect of the Capitol, J. George Stewart, with the approval of the House Office Building Commission, selected the firm of Harbeson, Hough, Livingston & Larson of Philadelphia to design a simplified, classical building in architectural harmony with other Capitol Hill structures. However, while the interior design of the other House Office Buildings retains decor one would expect to see in House Office Buildings (with cherry wood panellings, brass railings and marble floors), Rayburn possesses design style parallel to that of the 1960s, with chrome door handles, clocks, and elevators, loudly-colored walls coated with a shade of turquoise, and space-age fluorescent lighting fixtures.

The Capitol Subway System, an underground transportation system, connects the building to the Capitol, and pedestrian tunnels join it to all of the Congressional office buildings on Capitol Hill.

For construction of the Rayburn House Office Building, the Congressional bill appropriated $2 million plus "such additional sums as may be necessary." Such additional sums eventually totaled $88 million. Congressional leaders inserted a gymnasium into the building plans, a fact that was not publicly known at the time of construction. The gym is below the sub-basement level, in a level of the underground parking garage, and according to The Hill, a capitol hill newspaper, "features dozens of cardio machines outfitted with TV screens, an array of Cybex weightlifting machines and free weightS.