I agree, context can be very important. To an extent I’m somewhat fortunate because when I lived in NYC I worked in a lot of different ethnic restaurants and usually became good enough friends with most of the owners and staff that I was regularly invited to local cultural events.
However, regarding Seinfeld, I think it takes a lot more than understanding NYC Jewish culture. I was a room mate with a guy that was the cantor at his gay synagogue, went to all the after service get togethers and other events and parties, so I think I am somewhat versed in the NYC Jewish culture. However, I watched one episode of Seinfeld and today, 30 years later I still think it was half an hour I will never get back. I don’t think I’ve ever seen a show as pointless.