In very small conferences and workshops (with up to 50 submitted papers), reviewers are usually the members of the program committee. The papers are assigned manually by head organizer (Conference chair or chairs if there are more). Usually the chair knows the members of the program committee well and is able to take care of any conflicts of interest. Reviewers are also asked to declare their conflicts as soon as they get the papers to review (with implication that then the conflicted paper is reassigned).
In larger conferences, papers are assigned to tracks, headed by Area Chairs, who again do the similar job than conference chair at small conferences. In such conferences, Area Chairs may form the program committee, and the reviewers are other researchers in the field that get invited to review papers. Essentially, one level of hierarchy is added. In such conferences, paper matching is sometimes done automatically based on keywords supplied by the authors; in the same way, conflicts of interests are detected. One of such systems is Toronto paper matching system, used by many conferences:
https://openreview.net/forum?id=caynafZAnBafx