Computational Creativity (CS 185C)


Course Description

Computational Creativity is an emerging subfield of Artificial Intelligence that studies the potential for computers to be more than feature-rich tools, and instead to act as autonomous creators and co-creators in their own right. In a Computational Creativity system, the creativity comes from the machine, not the user, though in a hybrid system a joint impetus may come from both parties (http://prosecco-network.eu/node/242). Computational Creativity can also be defined as the computational analysis or synthesis of works of art, in a partially or fully automated way (Fernandez and Vico, 2013). In this course, we view this field through both of these perspectives, studying foundations of computer creativity and its applications to artistic domains, focusing on computational musicology, technology and dance, and computation in visual arts.

Course syllabus


Time and Location

Lectures
Time: MW 4.30PM-5.45PM
Location: Clark Building 243

Course Personnel

Intructor: Margareta Ackerman
Office hours: MW 3.30-4.30PM
margareta.ackerman[AT]sjsu[DOT]edu

Grader: Rathna Ramesh
rathna1993[AT]gmail[DOT]com

Mark Breakdown

Participation (including literature search reports): 25%
Assignments: 15%
Final Project (talk on previous work, presentation, and final report): 45%
Final Exam: 15%


Facebook Page

Please join the facebook page here.

Final exam:

Perform an evaluation of your final course project. While multiple methods of evaluation may be used, at least one method must be an approach that was taught in class (see below for notes on Evaluation of CC systems). The evaluation should center on the creativity of your final project, focusing on either the process that your system utilizes or artifacts made by your project, or both. Additional evaluation approaches may also be used. For example, if your project produces visual art, you may critically analyze the quality of the resulting art work.
The length of the writeup should be one page, single spaced with 11 or 12 size font. Up to two pages will be accepted. Bring a hard copy at the beginning of our final exam time, on May 23rd at 2:45.

Student Accomplishments:

Students have done an excellent job on their assignments this semester. Here are a few selected examples for each assignment.

Essay

This assignment asked students to write a short essay on a topic related to human or computer creativity.

Tempermental AI by Karan Bhargava.

Celtic Pagan Music saved by Technology, Creativity and Belief by Nityam Shrestha.

Creativity in Animal Kingdom by EJ Brand.

Twitter Bots

Students were asked to create a Twitter Bot, which produces some tweets that could have been made by a reasonably creative human.

DisneyPicAlive by Sunil Thapa and Miffy.
packages used - tweepy, markovify, textblob, PIL

Tiny Po by Nithyam and Joseph La Ferla.

Fortune Cookie by Karan and Ann.
Used Yerkee API to build the fortune database

SJRecruiter by Narbeh and Tom

Broken Knock Knock by Melvin McElrath, Chris Dal Porto, and Anthony Sokry

Troll-Moji5000 by Nathan Tseng and Un Hou Chan.

B&C Network by Colt Wilson and Barath


Conversion Assignment

Facial expression to Music conversion by Nityam Shrestha and EJ Brand. Emotions are extracted from photos and music is generated accordingly.

Image to poem conversion by Sunil Thapa and Miffy. Computer generates a poem based on the picture given to it and reads it with background music to the user.

Pictures to Abstract Art Collage by Jia Jun Wu, Joseph and Tom Lin. Combines random pictures (products from Amazon to photos of people) and creates abstract collage.

Poem to Image conversion by Un Hou Chan and Anthony Sokry. Poems are converted into pixel art that represent the mood of the poem.

Assignments

About you - Due 7th Feb
(If looking to add to course, due 1st Feb)


Email Rathna an easily printable one page document with the following :

1. A photo of yourself (headshot)
2. Your name
3. Why do you want to take this class?
4. Computing training and experience
5. Artistic training and experience
6. Favourite quote
7. Something memorable about yourself


Foundations of creativity essay - Due 22nd Feb by 4.29pm

Description: Write a 2-3 page essay (+ 1 page for references) on any topic relevant to creativity. Topics may be related to human or computer creativity. State a clear hypothesis and support it with sources as appropriate. Please select easily legible font size and margins, and bring a printed copy at the start of class on the due date.

Examples from previous course offerings:

Jason Chee's essay on music licenses and creativity and talk.

Nicolas Mauthes's essay on mobile culture and the decentralization of creativity.


Twitterbot Due March 6th, at the start of class.

Description: For a successful assignment, it is essential to come up with an interesting and original idea for your Twitterbot. You are encouraged to discuss your ideas with Maya. See the second set of lecture notes for further details of what makes a good Twitterbot.

To submit and present your work:

(1) Email all code and documentation, and a link to your online bot, to Rathna before class on the day of the due date. Feel free to have your bot follow @ackermanmaya so that Maya could follow your bot back. Make sure your Twitterbot posts sufficiently many Tweets before the deadline.

(2) Prepare a 15 minute presentation/demo on your TwitterBot to be shown in class on the due date. The presentation should address how the bot was made and include several interesting examples of its output. In addition, please prepare a demo of your bot, by having it produce new Tweets during class.

AWS hosting service for students.



Art conversation assignment Due on 5th April by 4.30pm

Description: Come up with a system that converts one art form into another. Examples include converting poetry to images, music into poetry, or photographs to music. The challenge is to retain some of the essence of the original work. For example, if you are converting melodies to abstract art, then we might expect a sad melody to be converted into an image that brings on similar feelings. Please feel free to discuss your ideas with Maya.

To submit and present your work:

(1) Email all code and documentation to Rathna before class on the day of the due date.

(2) Prepare a 15 minute presentation/demo on your assignment to be shown in class on the due date. In addition, please prepare a demo of your assignment, and be ready to run your program on new input during class.



Few examples of Art Conversion Assignments

Charles MacKay's audio visualization: Audio visualization using blender software along with mp3 ID3 tagging to identify BPM and key of song. Python script converts key into material color, by using a table of musical key characteristics to convert this into a color of similar traits. The beats per minute determines the particle velocity of the stars and clouds moving across the screen. The python script then creates 96 bars, and places them equally spaced radially around a circle. each bar divides the frequency of the song using the twelvth root of two as ratio to divide the upper and lower bound of the frequencies the bars will represent. The song's fcurve will then determine how the bars will be displaced depending on the energy in that frequency band.





Nicolas Mauthes's visual audio.



Nikita Ramesh's Painting to fashion: This art conversion project converts paintings to outfits. The ArtConverter Bot picks up a random painting from the database. It checks every pixel and gets its RGB color value. Then it converts the value into HSV. It will check the color of the pixel and increment that color's count. After all pixels are considered, the maximum three color values are picked. From these values, a top, a bottom and a pair of shoes are picked according to their color. The ArtConverter Bot then displays the combined image with hashtags of the maximum three colors.

Pankti Kansara's songs to images conversion project: presentation slides.



Project Proposal Due on 10th April by 4.30pm (hardcopy)

Description: Please prepare a one-page description of your course project. You're welcome to discuss it with Maya before the due date. Ensure that the project has sufficient depth to be suitable for a course project. Prior to submitting your proposal, please review the literature to ensure that your ideas are original - briefly discuss your review of related literature in your proposal if needed.



Literature review for your project Due on 17th April by 4.30pm (hardcopy)

Description: Perform a thorough literature review for your course project. Focus your survey on material that is closely related to your project. The purpose is twofold: (1) to discover ideas and methods that will help you in your project, and (2) to ensure that your work is original. Please bring a hardcopy of up to 3 pages including references.



Lecture Notes

Foundations of creativity

Computer Creativity

TwitterBots technicalities

Markov chains and genetic algorithms

Music Information Retrieval (Vidya's Lecture Resources)

Evaluation of CC systems

Computational Music Kitty's slides





Resources

Below is a list of hand-picked resources on computational creativity, focusing on the main topics that will be covered in this course. All of these resources are freely available online and are suitable for students new to this emerging area. These resources include academic writing in this domain, links to creative and co-creative systems, as well as videos and images of stunning works that were created using computational methods. Enjoy!

Human and computer creativity


Computational musicology


Computational creativity in visual art

  • The painting fool aims to be a creative artist in its own right.
  • AARON is a system that creates abstract computational visual art, including both interactive and independently created pieces.
  • Deep Forgeries is a TwitterBot that creates forgeries from your photos in the style of famous painters (user guide).
  • Drawing Apprentice is an AI drawing partner whose contributions are improvisational and based on the input of the user.
  • Step into the page, 3D drawing.

Technology and Dance


Miscellaneous


Related associations, journals, conferences


TwitterBot examples

Here are some Twitter Bot examples that were created in a previous offering of this course.
  • color_my_words, by Arturo Valery, identifies top news articles and creates abstract paintings that capture the emotions in their content.
  • digitaljimbot, by James Bach, JRR Tolkien and JK Rowling Remix Twitterbot. Here are the code, readme, and presentation.
  • wyisecrack, by Jacob Kattampilly, tweets random quotes and jokes related to computers by scraping the web.

Other resources

  • Markovify: a simple, extensible Markov chain generator (Python package).
  • Colour: a python package for convering and manipulating various color representations.
  • Pillow: Python imaging library
  • Sentiwordnet: sentiment analysis.
  • ssbot: Easy to use markov chains in a Google Spreadsheets.
  • nltk: Natural language toolkit.
  • PySynth: simple music synthesizer in Python.
  • MIDIUtil: a Python library that allows you to write muti-track Musical Instrument Digital Interface (MIDI) files from within Python programs.
  • Microsoft Emotions API : extracts emotions from human facial expressions.
  • Improviser : automatic music generation program.
  • Python Mingus : a package to make and analyse music.
  • pyGame : library for making multimedia applications like games built on top of SDL library.
  • Improviser : automatic music generation program.
  • Microsoft's CaptionBot : provides captions describing pictures fed to it.

www.000webhost.com