NHK Where is beauty? Grand Challenge

Scene Evaluation based on Aesthetic Quality

Automatic understanding of viewer’s impressions from image or video sequences is a very difficult task, but an interesting theme for study. Therefore, more and more researchers have investigated this theme recently. To achieve automatic understanding, various elemental features or techniques need to be used in a comprehensive manner, such as the balance of color or contrast, composition, audio, object recognition, and object motion. In addition, we might have to consider not only image features but also semantic features.

The task NHK sets is “Where is Beauty?”, which aims at automatically recognizing beautiful scenes in a set of video sequences. The important point of this task is “how to evaluate beauty using an engineering approach”, which is a challenging task involving human feelings. We will provide participants with approx. 1,000 clips of raw broadcast video footage, containing various categories such as creatures, landscape, and CGI. These video clips last about 1 min. Participants will have to evaluate the beautifulness of these videos automatically, and rank them in terms of beauty.

The proposed method will be evaluated on the basis of its originality and accuracy. We expect that participants will consider a diverse range of beauty, not only the balance of color but also composition, motion, audio, and other brand new features! The reliability and the diversity of the extracted beauty will be scored by using manually annotated data. In addition, if a short video composed of the highly ranked videos is submitted, it will be included in the evaluation.

INPUT

Raw broadcast video footage (about 1 min x 1,000 video clips)

Video format: MPEG4 (H.264), 640 x 360 pixels, 29.97fps
Audio: AAC 44.1 kHz stereo 1,411 kbps

OUTPUT

  • Manuscript that describes an algorithm to evaluate beauty.
  • Ranking list for all distributed video files in terms of beauty.
    The list should be described by the file name that we provide in csv format.
    Ranking lists submitted to us by June 10th will be evaluated against the ranking produced manually by 10 Japanese individuals. The result of this evaluation will be notified to the submitter before the paper submission deadline, to enable it to be included in the paper should authors so wish.

Recommended:

  • 1 to 2-minute short video that is composed of videos ranked in the top 10.
    Participants can manually edit the short video by using any shots in the top 10 videos.

EVALUATION

The three criteria for evaluation are in the following order of importance.

  1. Originality and adequacy of proposed algorithm
  2. Reliability and variety of ranking list in terms of beauty
  3. Beautifulness of the short video (if submitted)

 

We will provide a password to download the video clips only to those who sign our agreement form for handling the material data. If you are interested in this task, please contact us.

CONTACTS

Masaki Takahashi takahashi.m-iu -at- nhk.or.jp
Masanori Sano sano.m-fo -at- nhk.or.jp

Japan Broadcasting Corporation

Comments are closed.