Categories in perception and classification exists in many domains, including artificial shape perception, color discrimination, and lexical categories of words. Sound perception is no different, and various types of sounds have been explored for the type of categoricity effects they elicit. When we consider two classic cases of Stop Consonants and Vowels, they differ in the extend to which they are perceived categorically. Categorical perception has been proposed for the very categorical stop consonants, while the Perceptual Magnet Effect has been proposed for the much more continuous vowels. This, however, leaves open the question of whether we need separate accounts of how sound perception occurs to cover both cases. In our work, we use a Bayesian model of speech perception to explore factors that might cause this difference. Simulations show that perception of vowels and stop consonants, along with fricatives as an intermediate case, can all be captured under a single model in which listeners use their knowledge of phonetic categories to infer the sound that a speaker intended. This suggests that the differences in the way we perceive vowels and consonants, when viewed at the computational level, can be explained as parametric variation within a single framework. I conclude with implications for theories of sound perception and an overview of potential future work to expand on these ideas.