Turns out there is no filter that asks wether or not an imagefield (or filefield) has a value. I came across this tip on Drupal.org. The recommendation was to use a relationship, and then choose "Require this relationship" to act as a filter. Pretty cool.

So assuming your imagefield is just called "Image", here is the walkthrough:

1. Add a new view of type "node"
2. Add one or more fields (I assume you want to include the image field) in the normal way