Grapes, seeds, and stems

Answer

Stems: Unless there are grapes on the stem, the stem does not have kedushat shevi't and can be disposed.  Often, as you pointed out, there are little bits of fruit stuck to the stem. Unless you would eat them in regular years, they are not significant and secondary to the stem, thus they do not have kedushat shevi'it.

Grape seeds: Grape seeds are interesting since they are often eaten along with the grapes themselves, but not on their own. There are also people who do not eat the seeds at all.

I want to point out that it is perfectly safe to eat grape seeds (in moderation; those on blood thinners should consult their doctors). They are considered a rich source of antioxidants, flavonoids, and melatonin. They are believed to support heart and brain health, better sleep, and blood circulation. Thus, grapeseed oil, extracted from the seeds, is consumed for its many health benefits. 

HOWEVER, since this shemitah grape pits are not conventionally eaten on their own, they do not have kedushat shevi'it and can be thrown away regularly.

Of course, if in subsequent shemitot people begin eating grape pits like they do pumpkin seeds and such, their shemitah status will change.

Note that the same holds true for pomegranate seeds. While many eat the hard, white seeds along with the fleshy fruit, others spit out the seeds. In any case, the seeds are certainly not (conventionally) eaten on their own. While packed with nutrients, they are nevertheless are not currently considered food and thus will not have kedushat shevi'it when pomegranates (soon!) ripen this year.