You can deal with this through background correction of your spectra, which is a very common practice in spectrometry and implemented in any good spectrometry software.
In the most general sense, background correction consists of recording the spectrum of the spectrometer path without the analyte, and then subtracting that background spectrum from the spectrum of the analyte. In that sense, background correction is a form of difference spectroscopy. The spectrum you obtain from this ideally contains only the peaks attributable to the analyte.
A concrete example is subtracting the absorption spectrum of a solvent in a cuvette from an analyte. You would run a cuvette with solvent but without the analyte and use that as your background, followed by a cuvette with the analyte. Subtracting the first spectrum from the second would yield the signal attributable only to the analyte, assuming that the cuvettes, solvents and atmosphere between the two runs are negligibly different.
Obviously, there are a lot of potential confounds here that need to be accounted for:
Cuvettes have to be made of the same material, have the same dimensions (path length is important!) Cuvettes and such are often designed to be maximally transparent to radiation in the region of the spectrum of interest to maximise sensitivity. This is why they sell uber-expensive quartz cuvettes for UV/vis.
Solvents/substrates have to be of identical composition between runs. You also don't want a solvent or substrate that absorbs strongly in the spectral region of interest for similar reasons to the above.
Cuvettes have to be homogeneously clean - if one is dirty, greasy or scuffed and scatters light more than the other, it may ruin the results.
The atmosphere has to be the same between runs - I've heard stories about quantitative IR spectroscopy of $\ce{C=O}$ stretches being thrown out by an increase in $\ce{CO2}$ due to people breathing (how rude!). You can understand that researchers who look at atmospheric $\ce{CO2}$ concentration need very careful sampling and analysis protocols for reasons similar to this.