Coming from a vegan, I say it's not wrong and pointless...you have cut certain animal products from your diet, and that is more than what most people in this world do. I find it silly that people give pescatarians negative flack, because the way I see it, any progress in the name of animals is progresss nonetheless, and I respect you fully for even being semi-vegetarian; whether it be in the name of animal's rights or not.
I also believe that, while it is no doubt an inhumane industry, just as is human genocide and abusive regime's of countries, each person has the right to eat meat if they wish...and I believe I should accept this- NOT because of moral obligation, but because of the current social aspect this world views the meat industry in. Most don't think the industry is "too horrid" so they eat, and if they do, then they cringe a little at abuse and continue eating meat because it's too good to give up. However, I don't object to educating people about the issues at hand with animal industries...I want to, but on the other hand I respect that after they have seen the true nature, it is their free right to decide to still eat meat or not.
So, basically, thanks for being pescatarian, and don't let the negative views of others bother you.