No, but yes.
No, it won't really be "compost" in the true sense; as
Beth_B said, there's a cooking process. Compost cooks at 140-150° F., which would be too hot for the plant roots. But, yes, you can get worm castings,
which is also called "vermicompost," so I guess the answer is, um... sort of...
The first year we moved into this house, I finally had room to garden. After the expense of shovels and rakes and planters, and various other things, I said to heck with it,
I'm going cheap on the potting soil. The bigger planters, I filled the bottoms of them with chunks of dirt and grass ripped out from where I was putting the garden. I stuffed in handfuls of grass clippings. The planters I had sitting on the ground got filled with worms!!
I assume they crawled in through the drainage holes, going after the decomposing vegetation in the pots.
If you already have worms in the containers, you might as well feed them for the castings they'll provide. My advice is to run the food scraps through a blender (or at least some kind of chopper or processor,) before you put them in the container. You don't want it too watery, (don't drown the worms or your plants,) but you need some water to get them moving in the blender. You'll want it like a paste, try for the consistency of baby food - worms' mouths are very small, after all. Also, worms don't have stomachs, they have gizzards, like birds, so you'll need to provide them with some grit to keep things moving along.
Crushed up eggshells or coffee grounds would work.
I don't think you'd need to have brown material, since it's not actually compost. A lot of people that have worm bins put a layer of damp newspaper over the bin, so dampened shredded newspaper (or coffee filters) over the top of the container would probably help. It'd keep in moisture, and provide more darkness for the worms to stay active, anyway. And help keep out any flies. And help keep in any smell.