...and they nearly-unanimously insist that you change your favorite part. For the sake of discussion, let's pretend it's the ending. They don't like it. They feel that it's unexpected, and leaves the main plot question dangling. They all say it has to go.
What's a writer to do? If only a few readers felt this way it would be easy to ignore them, but with so many complaints it seems clear that the end isn't really working. You feel trapped between the proverbial rock and the hard place. Changing the story will destroy its soul, yet if you leave it alone it will likely languish, unsold, for decades to come.
This is the False Dichotomy of critique groups, and it used to be my nemesis. It seemed I always had to choose between changing a story or leaving untouched, and neither option felt right.
I finally realized that the dichotomy is an illusion. Here's the truth: If you wish to satisfy your audience with regard to a plot point, you must either (1) Change it, or (2) justify it.
See? It's not about 'change' vs 'don't change'. It's about 'change' vs 'justify'.
Your first reader didn't like the ending? You can either change it or lay the necessary groundwork to make the ending you've chosen feel right and proper. Your best friend doesn't think Julian should dump Margaret for Cynthia? Well, you can either hook Julian up with a different girl, or you can get the reader farther into Julian's head so his actions make more sense.
If your story feels right to you, it probably means you have information you neglected to share with the reader. Put it on paper. Justify your story, don't let it be cowed into inferiority by people who are missing half the data.