How do I lower the baseline of a picture bullet?

Okay, I think your Word version was reported incorrectly. If you have the Developer tab, then you definitely don't have Word Starter. I'm away from the computer where I created this, but I'll try to remember the steps I used:

1. I inserted a picture, sized it to "bullet size," and set the text wrapping to In Line With Text.
2. With the picture selected, I inserted a frame. I have an Insert Frame button on the QAT, but you can use the button in the Legacy Forms palette of the Legacy Tools in the Controls group on the Developer tab.
3. You'll need to remove the border from the frame, so select the frame (not the graphic in it) by clicking on the hashed border. You'll then be able to use the No Border button on the Borders dropdown in the Paragraph group on the Home tab.
4. I found that the graphic in the frame (which is now in its own paragraph) had the same formatting as the paragraph in which I'd inserted it. That happened to be Normal, so it had 1.15 line spacing and 10 pts Spacing After. I had to set it to Single with no spacing to make the text wrap correctly around the frame. You can also adjust the "Distance from Text" in the Format Frame dialog (double-click on the frame border to access that).
5. Once you've got the framed graphic sized and placed to your satisfaction, select it and press Alt+F3. This will open the Create New Building Block dialog with AutoText selected as the default. Type a name for the entry (you can also change the gallery if desired) and click OK.
6. You should be able to insert your bullet by typing the first four letters of the entry name and pressing Enter when you get the AutoComplete suggestion, but if that doesn't work, you can type a few letters and press F3 or use Insert | Text | Quick Parts | AutoText. The second bullet in my screen shot was inserted this way.

My example was just a rough-and-ready trial, but you should be able to duplicate it using your graphic. It's nowhere near as convenient as a true bulleted style, but at least it's a workaround.