Many people glue the lids on their learning (discovery) bottles. If I had very small children with which to use them, I would certainly glue them as well. But since my students use them during supervised times like during our Daily 5 block and Math centers, I do not glue the lids on. Instead I fasten the lids tightly. Cover the entire lid area with a circle of a pretty fabric and secure it with a pretty color zip tie. I make sure the zip tie is as tight as can be so the lid area is secure. (A note of caution about zip ties, they can be a bit sharp on the corners of a cut off part. For this reason, I snip off the tiny corners that stick out after cutting the zip tie to the desired length.)
Now, if I decide to change the contents of a bottle for some reason, I can do so fairly easily.
The zip ties can be covered with a ribbon or ponytail holder if you want. The bottle in the top picture that has the tens frame attached to it uses a ponytail holder to attach the tens frame to the bottle. That way the student can remove the frame and mark the items found with a dry erase marker to keep track. More on the specific bottles will be posted soon.