It only seems like evidence to people with common sense.
Common sence is just another word for assumptions
Not according to our courts. I have served on four juries, two criminal and two civil. In the judge's final instructions to the jury, they are told to draw logical inferences from the evidence. IOW, USE COMMON SENSE.
As it applies to this issue, had the case gone to trial, the jury would have been presented the evidence of the fiber matching. They would have had to ask themselves, how likely was it that the fibers on the butt plate of the rifle were from an identical shirt to the one Oswald was wearing when arrested. They would have to ask themselves how likely it was that the fibers from the bag were from a blanket identical to the one Oswald kept his rifle wrapped in. A logical inference is that the rifle fibers came from the shirt the rifle's owner was wearing when he was arrested. A logical inference would be that the bag fibers came from the blanket owned by the person whose prints were on the bag. When coupled with all the other evidence of Oswald's guilt, those are the only logical conclusions a reasonable person could reach.
If you want to cling to the theoretical possibility the fibers came from identical objects to the ones associated with Oswald, that is your right. Any thinking person who is interested in the truth of JFKA and is not determined to argue for Oswald's innocence, no matter how much evidence there is to the contrary, would understand how ridiculously unlikely those fibers came from objects not owned by Oswald.
The jury in the Wayne Williams case had to make similar judgements about the fiber evidence presented by the prosecution. They had no trouble finding him guilty and sending him to jail for the rest of his life, based primarily on the fiber evidence. They had no problem concluding there was no reasonable doubt that the fibers came from objects associated with Williams. They weren't assuming. They were making sound, common sense judgements. The fact that the spree of child murders in Atlanta ended when Williams was taken into custody vindicates their verdict.