Some people are so insecure about their writing that they have to justify it. Believing that what they write is somehow "better" than other genres is one way of them doing that.
Let me say this: every genre has its place. A story is a story is a story.
It doesn't matter if it's a literary novel or a western or a crime story or an erotic romance. If you write well, nothing else matters.
Writing is a venture on which everyone has an opinion. Some are based on knowing the publishing world, on real experience. Others just go on hearsay and the phrase "Well, everyone knows that..."
Being a writer means shifting through these opinions for the ones that are worthwhile. Listening to the naysayers will get you down and have a detrimental effect on your blood pressure. Such people will make sweeping statements based on very little evidence. Ignore them.
Those you need to listen to are those that believe every story is a worthwhile tale, that believe others will find it so also.
Your novel is not going to change the world, but how many books really have? If it touches one person - just one - then you've achieved something great.