Imagine you are constructing regular polyhedra from rigid equilateral triangles, squares, and regular pentagons. Adjacent faces will be connected edge-to-edge by freely-swinging hinges. The minimum number of faces of a regular tetrahedron one needs to put in place to produce a rigid structure is three: two faces can clearly swing, but three faces make a triangular pyramid which is rigid (because the missing face is constrained to be an equilateral triangle).
For a cube or dodecahedron, again connecting all three faces that surround one vertex is rigid by similar reasoning, so the minimum is three for these polyhedra as well.
For an octahedron, however, it's not difficult to convince yourself by comparing the octahedron and the trigonal or pentagonal bipyramid (or by building a paper model) that whenever two adjacent faces are missing, the resulting model will not be rigid. Thus, the minimum number of faces required to create a rigid model is six (delete two opposite faces).
What's the corresponding minimum for a regular icosahedron?
EDIT
There are two cases, according to whether vertices are considered to be connected (with free pivots) as well.
In the case without vertex connections, the minimum is 16. To see this, note that the icosahedron with two edge-adjacent faces missing is not rigid (as again a paper model quickly verifies). On the other hand, if two faces sharing a vertex but not an edge are missing, then the face between them is only attached at one edge and can swing. Thus, for the configuration to be rigid, no two missing faces can share a vertex. Since there are twelve vertices, that means at most four faces can be missing for rigidity. On the other hand, deleting four faces whose centers make a regular tetrahedron produces a rigid structure. So the minimum is 16.
The middle step of the above argument fails if vertices are also considered to be attached (and pivot freely), suggesting there might be a rigid structure with fewer faces in that case.
In the case with vertex connections, the minimum for an octahedron is actually four, produced by using every other face.