Alyssa,
The filter looks black because it filters out most of the light. But when you look at the Sun, it is SO bright that some of the sun's light makes it through the filter. It looks red (not yellow) because more of the yellow and other colors are filtered out than the red.
Do you have red-blue glasses? Look at a candle or light blub through one or the other. What color is the candle?
Paulett