Italian Giallo Films seem to be having a little resurgence recently. I first was exposed to them when the Criterion Channel app had a collection of them for Halloween a few years ago and now watch a few of them around this time of year mixed in with other spooky season movies. They are terrible movies, but are usually visually striking. It’s like the cinematographer or director is competing to make the most saturated, dramatic and elaborate scene possible. Coupled with the 1960/70s aesthetic they are really kind of fun to visually experience. Watching them you notice recurring tropes, things like extreme zoom shots of the actors eyes, a shadow creeping up a staircase or a POV knife wielding gloved hand. I was wondering if you had a large corpus could you automatically group these tropes and maybe discover new ones or even see influences?
Image Similarity algorithms are a pretty common approach to work with a corpus to find visually similarity but I wanted to group the content of the scenes not just how they looked. I want to group two scenes together of a shadow walking down a hallway even if they are visually very different. So I thought I could maybe use a vision language model to describe a scene and then use that textual description to group and explore. I decided to try and use the Qwen2.5-VL-72B-Instruct model and send it a frame from the movie every 5 seconds to build a textual representation of the film. I did this for about 70 Giallo films and used those texts to do some experiments. This post looks at that process and results.
All the code and data that I can make public is in this repo. This project is working with copyright material though and also it's pretty graphic so I’m limited in what I can realistically put up in the form of public tools and visualizations. But everything I created from the material such as scene descriptions and embeddings I put in the repository.
In any project collecting the data is often the hardest part. If you want to algorithmically examine Giallo films you need a lot of Giallo films. Wikipedia has a helpful list of films I decided to just focus on the 1970s films which really represent the peak I think of the genre. I spent some time gathering the ~140 films from the list as best I could. I ended up only able to get 70-ish of them in good quality. I was pretty discerning, a lot of these are on free streaming sites but they are in potato quality. However even only 70 films if you extract a frame every 5 seconds ended up being over 80,000 images. I thought this was enough to get a sense if it would work at all. Here is a list of the the 140 films I tried to get and which ones I was able to find:
I ran all 80k images through the model with a prompt asking it to identify things in the frame, striking features of the scene and a few sentence description. Here is an example of what a single frame data looks like:
"The.Bird.With.The.Crystal.Plumage.1970.mkv__0465.jpg": {
"tags": [
"man",
"glasses",
"telephone",
"hand",
"face"
],
"striking": [
"closeup of eyes",
"dark lighting",
"intense gaze"
],
"description": "Man holding telephone to ear. Eyes focused and serious. Dark shadows cast on face.",
"file_name": "The.Bird.With.The.Crystal.Plumage.1970.mkv__0465.jpg"
}
In this case the count in the filename 0465 is the timestamp of the film 0465*5 seconds == 38 Minute and 45 Seconds
At this point I built some tools that lets you explore the 70 film corpus textually by features. You can see across all film scenes that have "closeup of eyes" for example. Here is a video showing that interface if you are interested since I cannot put it up publicly (source code in repo though).
This tool lets you build clips from selected scenes, so you can make a supercut from the 70 films like this:
Or an exploration of reel-to-reel technology depicted in Giallo films:
I would make more but it is pretty annoying posting copyrighted material on YouTube, even short clips like this get flagged with copyright issues and there is no telling if the copyright holder’s policy is to completely block your video or simply monetize it.
Next step was to send the 80K frames to build embeddings. I used the Google gemini-embedding-001 in “cluster” mode and only used the “tags” and “description” section of the text to build the embedding.
Now with embeddings available I first built nearest neighbors for all the frames and created a little interface to let me manually browse those, again I can't put it online so here is video if interested.
But I wanted to automatically find tropes, I’ve built clusters before after reducing the 3072 dimensions using something like UMAP but I wanted to try building clusters without reducing dimensionality. I tried HDBSCAN which is interesting because it doesn’t force every vector into a cluster which makes sense for this application I wouldn’t expect every frame to be related to another.
The result was around 330 clusters which I sent off the descriptions of each cluster to a LLM to organize them into a single description of what all the descriptions had in common. The result is a pretty interesting list of possible scenes you would expect to find in a Giallo:
| # Frames | Cluster Name |
|---|---|
| 333 | Fearful or suspenseful women's faces and eyes dramatically obscured or revealed by darkness and shadow. |
| 309 | Intimate bedroom scenes of couples in close physical embrace or repose, typically under soft or dim lighting. |
| 290 | Tense, dimly lit indoor dialogues between two individuals. |
| 186 | Movie credit sequences featuring high-contrast text against simple, often dark or solid-colored backgrounds. |
| 154 | Suspenseful nocturnal scenes featuring solitary vehicles navigating or parked on dark, desolate, or forested roads, often with headlights piercing the pervasive shadows. |
| 118 | Scenes of men engaged in tense and urgent phone calls within dimly lit, suspenseful indoor settings. |
| 117 | Tense, dimly lit nocturnal conversations between a man and a woman inside a car. |
| 116 | Vulnerable women sleeping in dimly lit, suspenseful settings. |
| 106 | Tense and intimate dialogues or confrontations among women in private indoor settings. |
| 101 | Contemplative and investigative office scenes. |
| 100 | Eerie nocturnal scenes set in dense, dark forests, featuring shadowy figures and an atmosphere of hidden dangers. |
| 100 | Ominous encounters with shadowy figures at dimly lit doorways. |
| 98 | Professional or investigative office meetings among men engaged in serious or tense discussions. |
| 97 | Suspenseful scenes of women cautiously engaging with dark or shadowy doorways. |
| 96 | Close-up shots of a hand gripping a knife in darkness, establishing intense suspense and the imminent threat of violence. |
| 95 | Dark, suspenseful scenes of an unidentifiable figure emerging from profound obscurity. |
| 86 | Tense domestic interactions or conversations in a kitchen setting. |
| 83 | Tense outdoor discussions between two men, frequently set in lush natural environments or against rustic stone backdrops. |
| 76 | Suspenseful scenes depicting women fearfully traversing dimly lit staircases and hallways, marked by pervasive shadows creating an eerie and foreboding atmosphere. |
| 72 | Scenes of intimate embraces and kisses between a man and a woman, often portrayed in atmospheric low light or dramatic color. |
| 68 | Contemplative formally-dressed men isolated within serene yet mysterious natural outdoor environments. |
| 66 | Dimly lit scenes of vulnerable men in bed. |
| 65 | Shadowy figures navigating dimly lit, suspenseful staircases. |
| 64 | Giallo scenes of a woman being stalked by a masked, knife-wielding figure in a shadowy corridor. |
| 61 | Scientific investigation and discussion in laboratory settings. |
| 60 | Pivotal, tense discussions or confrontations between formally dressed men in private settings. |
| 59 | Courtroom trial proceedings with judges, lawyers, and juries. |
| 57 | Intimate scenes depicting vulnerable women at rest or asleep in dimly lit beds. |
| 55 | Dramatic close-ups of women's faces conveying intense emotional states. |
| 54 | Detailed close-ups of analog tape recorders emphasizing their mechanical operation. |
| 54 | Close-up depictions of men with bloodied faces exhibiting intense pain and distress. |
| 52 | Dramatic chiaroscuro close-ups of men's faces conveying intense emotion and suspense. |
| 50 | Darkly atmospheric giallo tableaux depicting the motionless, often bloodied body of a woman, typically with a knife nearby. |
| 49 | Tense reveals of a knife-wielding, shadowy assailant in a dark setting. |
| 48 | Serious Outdoor Conversations between a Man and a Woman. |
| 48 | Terrorized women in darkened bedrooms facing unseen or impending threats. |
| 47 | Scenes depicting formally dressed men engaged in tense, often dimly lit, indoor conversations or deep contemplation. |
| 46 | Solemn or mysterious scenes featuring clergy, often engaged in rituals, within dimly lit religious or ornate settings. |
| 45 | Pensive scenes of women smoking in dark, mysterious interiors. |
| 44 | Dimly lit dining scenes featuring intimate or tense conversations. |
| 43 | Scenes featuring women in contemplative or cautious states amidst dense natural surroundings. |
| 43 | Scenes of quiet intimacy between a man and woman in bed. |
| 41 | Tense exchanges with clergy in dimly lit ecclesiastical settings. |
| 41 | Masked killer attacks on terrified women, highlighted by blood on white clothing. |
| 40 | Dark, reflective nocturnal waterscapes conveying suspense and hidden threats. |
| 40 | Dramatically lit portraits of men in suits engaged in intense, contemplative, or suspicious thought. |
| 39 | Dimly lit nocturnal car interiors focusing on a woman's tense or contemplative state. |
| 38 | Intimate or tense bedroom scenes featuring smoking couples. |
| 36 | Winding road sequences featuring solitary cars traversing isolated natural landscapes. |
| 36 | Motionless, bloodied male victims in dark, tense settings. |
| 34 | Mirror reflections conveying suspense and impending danger to women. |
| 33 | Opening and closing credit sequences overlaid on evocative natural landscapes, frequently depicting trees or water at dusk or twilight. |
| 33 | Funeral services and processions. |
| 33 | Suspenseful nocturnal scenes featuring a lone figure navigating dimly lit, shadow-laden urban environments. |
| 32 | Film credit sequences featuring dramatic nighttime vehicular destruction. |
| 32 | Serious or urgent discussions between a nurse and a man, predominantly in medical or clinical settings. |
| 32 | Somber or eerie cemetery scenes featuring male figures. |
| 32 | Dimly lit close-ups of a man's intense, contemplative, or suspicious expression, often against a dark background. |
| 32 | Scenes of a masked assailant menacing a terrified woman with a knife. |
| 31 | Atmospheric shower scenes depicting vulnerable women, often emphasizing intimacy and tension. |
| 31 | Eerie nocturnal urban scenes featuring a solitary male figure. |
| 31 | Shadowy close-ups of hands manipulating door mechanisms in tense, dark environments. |
| 30 | Scenes depicting a woman's tense introspection or contemplation within a mysterious, dimly lit interior. |
| 29 | Scenes depicting studio photography sessions, often featuring models and photographers. |
| 29 | Eerie nocturnal cemetery and graveyard scenes. |
| 29 | Scenes of women engaged in tense or distressing phone calls within dimly lit, suspenseful interior settings. |
| 29 | Tense, low-light outdoor conversations between a man and a woman. |
| 28 | Scenes featuring various aircraft in flight or at airport locations. |
| 28 | Tense outdoor dialogues between two women in natural, serene environments. |
| 28 | Intense conversations between a man and a woman at lively parties or nightclubs. |
| 28 | Tense nocturnal movements of solitary women through dimly lit, shadow-filled environments. |
| 27 | Bustling urban street and market scenes. |
| 27 | Scenes featuring solitary women in contemplative or pensive moments, primarily on benches in serene park or garden environments. |
| 27 | Solemn cemetery visitations and grave-side interactions. |
| 27 | Scenes of women in bathrooms engaged in personal grooming, self-examination, or contemplative reflection. |
| 27 | Solitary, contemplative smoking scenes in dimly lit, shadowy settings. |
| 27 | Serious office discussions at a desk. |
| 27 | Tense group discussions in affluent or intellectual interior settings. |
| 26 | Opening credit sequences featuring airplanes and production details. |
| 26 | Rugged and isolated coastal seascapes. |
| 26 | Vibrant stage performances, primarily by women, in nightclub or cabaret environments. |
| 26 | Serious conversations between women in indoor settings. |
| 26 | Eerie nocturnal exteriors of secluded houses. |
| 25 | Tense, dimly lit conversations between characters, frequently featuring smoking. |
| 25 | Contemplative indoor scenes of men, frequently by windows or curtains, conveying a mood of serious introspection and quiet tension. |
| 25 | A lone figure's tense or contemplative presence within an eerie, shadowed forest. |
| 25 | Scenes characterized by overwhelming darkness and obscured forms, hinting at an unseen lurking menace. |
| 25 | Scenes of a masked killer menacing a terrified victim with a knife. |
| 24 | Dusk-lit seaside conversations and intimate interactions. |
| 24 | Close-up shots of women displaying intense or contemplative expressions. |
| 23 | Newspaper headlines and articles detailing murder investigations and accusations. |
| 23 | Forensic examination of a deceased body in a clinical setting. |
| 23 | Tense and mysterious scenes of men in dimly lit cars at night. |
| 23 | Panicked female flight through an ominous forest. |
| 23 | Private conversations between a man and a woman on a couch. |
| 23 | Atmospheric scenes depicting a masked assailant's impending attack on a woman with a knife. |
| 22 | Equestrian and carriage scenes set in diverse natural and estate landscapes, often blending serenity with underlying mystery. |
| 22 | Historic Venetian canal scenes. |
| 22 | Tense armed confrontations in dimly lit interior settings. |
| 22 | Intimate and vulnerable scenes of nude or partially undressed women interacting with mirrors in private, dimly lit settings. |
| 22 | Tense, elegant dialogues and confrontations in opulent, often theatrical, settings. |
| 21 | Scenes depicting conversations between two women in a maritime or coastal environment. |
| 21 | Scenes depicting watercraft in transit with prominent distant geographical features. |
| 21 | Hospital scenes featuring men with bandaged head injuries and specialized medical equipment. |
| 21 | Suspenseful portrayals of men, often shirtless, in dramatically lit, confined interior spaces. |
| 21 | Eerie nighttime graveyard scenes featuring women experiencing fear or grief. |
| 21 | Detailed portrayals of women engaged in desk-based intellectual or administrative work. |
| 21 | Tense, dimly lit armed confrontations and standoffs between men. |
| 21 | Dimly lit bar and nightclub dialogues between a man and a woman. |
| 21 | Strategic contemplation scenes featuring a man at a desk with chess pieces and a map. |
| 21 | Tense or intimate conversations set within vibrantly patterned and intricately decorated interior spaces. |
| 21 | Intimate and tense face-to-face encounters in dim, shadowy lighting. |
| 21 | Dramatic, low-lit close-ups of men with intense, contemplative, or urgent expressions. |
| 21 | Suspenseful scenes of shadowy figures partially obscured by curtains in dim lighting. |
| 21 | Scenes of contemplative men in suits within indoor environments featuring curtains. |
| 21 | Pensive indoor scenes of suited men, often near a door. |
| 21 | Perilous navigation of shadowed staircases. |
| 21 | Masked assailant knife attack scenes. |
| 20 | Bustling market scenes depicting people shopping, interacting, and browsing various goods. |
| 20 | Hospital scenes featuring medical examinations and concerned observation of patients. |
| 20 | Tense conversations between two women in a car's backseat. |
| 20 | Contemplative or emotionally conflicted nuns. |
| 20 | Vehicle-bound scenes of serious dialogue or contemplative tension. |
| 20 | Introspective or apprehensive moments featuring a woman in a dimly lit bedroom. |
| 20 | Knife attack on a woman. |
| 20 | Contemplative close-ups of men in formal wear deep in thought, often featuring subtle indoor greenery. |
| 20 | Contemplative close-ups of men in formal attire. |
| 19 | Bustling Train Station Scenes of Travel and Anticipation. |
| 19 | Opening credit sequences set over scenic landscapes and driving footage. |
| 19 | Tense, shadowy close-ups of a hand gripping a gun, portending an imminent threat. |
| 19 | Close-ups of women in states of intense fear and distress, marked by tears and dramatic lighting. |
| 19 | Tense, shadow-laden hallway scenes featuring a woman navigating an ominous atmosphere. |
| 19 | Giallo scenes featuring a woman in a dark forest with a glinting knife. |
| 18 | Architectural establishing shots of imposing urban and historical building exteriors. |
| 18 | Photography and film studio sessions featuring models and motorcycles. |
| 18 | Theatrical stage performances featuring dancers or performers in elaborate, often feathered, costumes under vibrant lighting. |
| 18 | Leisurely or contemplative boat scenes with a man and a woman. |
| 18 | Serene and mysterious scenes of women interacting with boats in natural, vegetated waterscapes. |
| 18 | Subdued indoor scenes of male characters in pensive thought or focused conversation. |
| 18 | Tense official questioning or police-involved confrontations. |
| 18 | Scenes of tense, asymmetrical discussions between three men, typically in an intellectual or professional indoor environment. |
| 18 | Scenes of two men having tense, often smoke-filled, discussions in a cluttered office or study. |
| 18 | Intense contemplative and investigative dialogues among men. |
| 17 | Scenes of women swimming, floating, or struggling in various bodies of water. |
| 17 | Outdoor dialogue scenes in the rain featuring characters walking with umbrellas. |
| 17 | Suspenseful indoor scenes featuring characters in wheelchairs. |
| 17 | Somber graveyard encounters between a formally dressed man and woman. |
| 17 | Ominous foot and leg shots with prominent shadows. |
| 17 | Scenes of women in their distinctively decorated private domestic spaces. |
| 17 | Clandestine outdoor meetings between two men. |
| 17 | Intimate portrayals of a woman's anxious contemplation or fearful anticipation in a tense indoor environment. |
| 17 | Introspective scenes of women gazing at their reflections in mirrors. |
| 17 | Scenes of ominous lurking figures or concealed threats, dramatically illuminated by faint window light. |
| 17 | Tense indoor discussions between two men. |
| 17 | Eerie, suspenseful scenes featuring women's silhouettes by windows in darkness. |
| 17 | Scenes depicting an obscured threat poised at a doorway. |
| 16 | Motorcycle journeys on winding, verdant rural or mountain roads. |
| 16 | Scenes of nurses attending to patients in hospital rooms. |
| 16 | Intimate bathing scenes between a man and a woman. |
| 16 | Tense scenes depicting hands interacting with old-fashioned telephones in darkness. |
| 16 | Eerie, candlelit church and altar scenes featuring solemn or mysterious figures. |
| 16 | Scenes depicting tense and secretive discussions within elegant library settings. |
| 16 | Tense conversations between a man and a woman inside a car. |
| 16 | Indoor scenes of formally dressed men in serious contemplation, often featuring prominent paintings or textured walls as backdrops. |
| 16 | Chiaroscuro portraits of enigmatic male figures. |
| 16 | Ominous Silhouettes in Doorways |
| 15 | Scenes of children playing in diverse outdoor environments. |
| 15 | The scenes depict competitive motocross racing events. |
| 15 | Nocturnal police investigations and patrols conducted on urban canals. |
| 15 | Interactions among diverse passengers on a bus during travel. |
| 15 | Vibrant outdoor poolside scenes depicting leisure, conversation, and observation. |
| 15 | Serious classroom discussions between a man and a woman. |
| 15 | Portraits of nurses displaying focused concern or contemplation. |
| 15 | Ominous scenes of women struggling or submerged in dark waters. |
| 15 | Pensive female gazes from a car window. |
| 15 | Intimate mealtime conversations between a man and a woman. |
| 15 | Women engaged in private telephone interactions from bed. |
| 15 | Scenes depicting women in contemplative or apprehensive repose on couches. |
| 15 | Intimate and often tense bedroom scenes featuring a woman and a man amidst ornate decor. |
| 15 | Tense and critical conversations between a man and a woman inside a car. |
| 15 | Lurking shadowy figures silhouetted against dimly lit walls. |
| 15 | Private bedroom encounters between a man and a woman, frequently involving tension or serious discussion. |
| 15 | The scenes depict elegant, private conversations between a man and a woman in opulent indoor settings. |
| 15 | Intimate, serious discussions between two men. |
| 15 | A woman covertly observing through curtains or shadows, conveying fear and suspense. |
| 15 | Stylized masked killer murder scenes. |
| 14 | Pharmacy and clinical environment scenes depicting characters, interactions, and pharmaceutical items. |
| 14 | Tense forest scenes of pursuit and confrontation. |
| 14 | Establishing shots of mysterious, often isolated, castles or grand estates at night. |
| 14 | Discussions between men in urban or coastal waterside settings. |
| 14 | Solemn or tense discussions between two men in a graveyard. |
| 14 | Isolated figures, typically women, in vast and rugged coastal settings, often at dusk or night. |
| 14 | Contemplative male figures in atmospherically lit interior settings. |
| 14 | Tense police interrogation scenes. |
| 14 | Intimate interior scenes of women observing ornate decorative art and objects. |
| 14 | Ominous or suspenseful close-ups of solitary footwear in darkness. |
| 14 | Eerie candlelit scenes emphasizing looming shadows and suspense in pervasive darkness. |
| 14 | Intimate and vulnerable moments of men in repose. |
| 14 | Intimate and tender physical contact between individuals. |
| 14 | Scenes of men engaged in serious discussions while traveling by car. |
| 14 | Giallo murder tableaux of female victims. |
| 14 | Intimate and tense emotional exchanges between a man and a woman. |
| 14 | Scenes of women poised in doorways, exhibiting tense contemplation or watchful suspense. |
| 14 | Eerie giallo interior scenes featuring a cautious woman at a threshold. |
| 14 | A man's solitary, shadowed contemplation or troubled thought in a tense atmosphere. |
| 14 | Pivotal library conversations between two men. |
| 14 | Scenes depicting tense, formal dialogues between two men, invariably framed with one's back to the camera. |
| 14 | Scenes depicting solitary figures cautiously navigating suspenseful, shadow-filled hallways. |
| 14 | Impending knife attacks on a vulnerable woman in a darkened bedroom setting. |
| 13 | Outdoor credit sequences featuring a man in a natural setting. |
| 13 | Stylized credit sequences set against bustling urban streets with a prominent web-like or spiderweb overlay. |
| 13 | Elevated panoramic views of urban cityscapes and rooftops. |
| 13 | Scenes of intricate photographic operations conducted in darkness or low light. |
| 13 | Giallo film title sequences featuring a partially obscured woman against a vivid red background. |
| 13 | Scenes of urgent and frantic chases or escapes. |
| 13 | Serene yet foreboding waterside landscapes at dusk, framed by silhouetted trees. |
| 13 | Scenes depicting a woman, often with a man, on or by a bridge over calm water, framed by urban or ancient landscapes. |
| 13 | Bustling vintage urban street and square scenes with traffic and pedestrians. |
| 13 | Tense two-person conversations on atmospheric sea cliffs. |
| 13 | Men in suits navigating urban street environments. |
| 13 | Intimate or tense bathtub encounters between a man and a woman. |
| 13 | Injured men, often in suits, slumped or pinned against walls. |
| 13 | Scenes depicting a man driving a car on a road with a focused demeanor, traversing diverse natural and urban environments. |
| 13 | Scenes of characters navigating dimly lit spaces filled with mannequins or veiled figures, creating an eerie and suspenseful atmosphere. |
| 13 | Tense police-involved investigative discussions or confrontations. |
| 13 | Scenes of men drinking contemplatively in sumptuously decorated, atmospheric interiors. |
| 13 | Tense, dimly lit street conversations between men, often observed by or involving a woman. |
| 13 | Men conducting desk-based investigation or journalistic work in dimly lit office settings. |
| 13 | Flashlight-guided investigations within dark, mysterious, or ominous environments. |
| 13 | Contemplative female portraits. |
| 13 | Suspenseful nocturnal approaches to isolated, dimly lit houses. |
| 13 | Tense and serious hallway discussions between a man and a woman. |
| 13 | Dimly lit bedroom scenes depicting resting couples, characterized by deep shadows and an atmosphere of quiet tension and vulnerability. |
| 12 | Tension-building bathroom scenes featuring dripping or flowing water and ominous shadows. |
| 12 | Scenes depicting the atmospheric operation of film projectors and reels in dark or dimly lit environments. |
| 12 | Suspenseful nocturnal and dimly lit train sequences. |
| 12 | Scenes depicting hands interacting with handwritten documents or correspondence. |
| 12 | Secluded vintage car scenes in natural, often wooded, environments. |
| 12 | Enigmatic felines with prominent, glowing eyes in dark, suspenseful settings. |
| 12 | Intense fires illuminating dark settings and casting dramatic shadows. |
| 12 | Hospital room scenes centered on investigative documentation or critical information disclosure around a patient's bedside. |
| 12 | Outdoor breakfast scenes between a man and a woman, marked by serious discussion or contemplation. |
| 12 | Crowded urban street scenes depicting everyday life. |
| 12 | Scenes of a man's shaving routine, often highlighting his reflection in a decorative bathroom. |
| 12 | Unsettling nighttime scenes of women dramatically illuminated by fires. |
| 12 | Tense, foggy forest confrontations between two men, often involving a firearm. |
| 12 | Scenes depicting a bloodied, injured man in a forest. |
| 12 | Characters in intimate and vulnerable bath scenes. |
| 12 | Establishing shots of mysterious, often ornate, buildings at night. |
| 12 | Tense, secretive conversations between men in dimly lit bar settings. |
| 12 | Intimate or emotionally charged embraces between a man and a woman, frequently against a door. |
| 12 | Tableaux of motionless women in outdoor settings dominated by a distant castle. |
| 12 | Scenes of a contemplative mustachioed man in a library setting. |
| 12 | Scenes of women engaged in tense or concerned phone calls in intimate indoor environments. |
| 12 | Suspenseful scenes of female fear and apprehension in dimly lit indoor settings. |
| 12 | Pensive female character close-ups. |
| 12 | Enigmatic portraits of veiled women. |
| 12 | Tense power dynamic dialogues between two men in opulent, antique settings. |
| 12 | Dramatic close-ups of a man's face displaying intense, deep emotion under shadowy lighting. |
| 12 | Intense female gazes with serious or contemplative expressions against a dark, dramatic background. |
| 12 | Tense, private two-man discussions often occurring by doorways or in secretive indoor settings. |
| 12 | Shadow-obscured men in suspenseful doorways. |
| 12 | Scenes depicting women's terrified reactions and desperate gestures within pervasive darkness. |
| 12 | Tense scenes of a woman's hand reaching for a door handle amidst encompassing darkness. |
| 12 | Suspenseful approaches to dark doorways, hinting at an unseen or imminent threat. |
| 12 | Scenes of masked assailants stalking fleeing victims with an imminent knife attack. |
| 11 | Waterfront credit sequences often featuring figures and industrial or cityscape backdrops. |
| 11 | Silent, contemplative children illuminated against dark, eerie backdrops. |
| 11 | Giallo scenes featuring shirtless men in tense phone calls within distinctive home interiors. |
| 11 | Scenes of women gagged and restrained in distress. |
| 11 | A man's desperate confinement behind bars. |
| 11 | Scenes of a gathering around a critical or deceased individual in a tense, private setting. |
| 11 | Courtroom drama scenes. |
| 11 | Tranquil and secluded scenes featuring a man in a small boat on calm water surrounded by dense reeds and vegetation. |
| 11 | Leisurely outdoor meal and conversation scenes at private estates. |
| 11 | Conversations between a man and a woman in an airport setting. |
| 11 | Tense interpersonal confrontations between a man and a woman in elevators or confined spaces. |
| 11 | Cluttered kitchen confrontations between two men. |
| 11 | Dimly lit intimate encounters in cars. |
| 11 | Elegant indoor scenes depicting a man and woman interacting by or at a piano. |
| 11 | Dimly lit, shadowy car interior scenes establishing a mood of suspense and impending danger. |
| 11 | Tense or mysterious indoor phone call scenes featuring a man and an accompanying, reacting woman. |
| 11 | Serious outdoor conversations between two formally dressed men, often by a parked car. |
| 11 | Walking conversations between a man and a woman on streets. |
| 11 | Contemplative indoor close-ups of a blonde woman. |
| 11 | A distressed woman's suspenseful phone call in a dimly lit, tense environment. |
| 11 | Scenes of women in tense contemplation or ominous anticipation within dimly lit interiors. |
| 11 | Tense parlor or drawing-room scenes centered around tea service and strained conversations between a woman and men. |
| 11 | Contemplative indoor smoking scenes featuring men in suits. |
| 11 | Plot-critical outdoor exchanges between a man and a woman. |
| 11 | Hand-centric suspense sequences in deep shadow. |
| 11 | A man's tense and contemplative introspection in a formal, dimly lit indoor setting. |
| 11 | Tense discussions among individuals within opulent interiors featuring grand staircases. |
| 11 | Tense candlelit scenes of a woman in darkness. |
| 11 | Tense, shadow-filled alley scenes depicting an isolated figure encountering an emerging threat. |
| 11 | Dimly lit scenes emphasizing a figure's prominent shadow or silhouette against a wall. |
| 11 | Scenes of fallen women in dimly lit or shadowy environments. |
| 11 | Pensive male contemplation in shadowy, suspenseful interiors. |
| 11 | Intense close-up shots of blonde women's emotive gazes. |
| 10 | Scenes of intense outdoor surveillance using binoculars. |
| 10 | Suspenseful nocturnal scenes depicting wheelchair-bound figures navigating dark, outdoor environments. |
| 10 | Autopsy scenes or unsettling post-mortem examinations in giallo films. |
| 10 | Establishing shots of foreboding and isolated gothic castles. |
| 10 | Tense nocturnal searches and navigations through fog, illuminated by flashlights. |
| 10 | Tense conversations between a man and a woman aboard a boat. |
| 10 | Mirrored discussions between two women. |
| 10 | Urgent driving sequences. |
| 10 | Atmospheric scenes of men in dimly lit, cluttered kitchens conveying suspense and mystery. |
| 10 | Scenes depicting women in atmospheric interiors defined by colorful lamps and wall posters. |
| 10 | Dimly lit restaurant scenes featuring women smoking during conversations. |
| 10 | Eerie nocturnal urban or architectural credit sequences. |
| 10 | Scenes depicting a professional in a white coat engaged in a serious and concerned phone call. |
| 10 | Bathtub murder scenes. |
| 10 | Contemplative or tense scenes of a man in a car during rain. |
| 10 | Tense social gatherings in stylishly decorated living rooms. |
| 10 | Elegant outdoor social gatherings featuring two men in serious conversation, often framed by classical architecture. |
| 10 | Canal-side discussions between a man and a woman. |
| 10 | Intense close-up shots of women in hats conveying mystery through their gaze. |
| 10 | Tense, candlelit dinner discussions between two men. |
| 10 | Quiet, contemplative domestic scenes of a woman reading beside a reclining man. |
| 10 | Nighttime, dimly lit conversations between a man and a woman, typically by a car, characterized by seriousness or tension. |
| 10 | Clandestine and tense nocturnal encounters between a man and a woman in isolated forest settings. |
| 10 | Intimate bedside conversations. |
| 10 | Tense phone call scenes featuring men at dimly lit office desks. |
| 10 | Tense interpersonal confrontations in dimly lit spaces. |
| 10 | Contemplative and tense indoor scenes featuring a man by a door. |
| 10 | Shadowy and tense staircase scenes featuring a solitary male figure. |
| 10 | Moments of shadowy male apprehension or intense contemplation. |
| 10 | Tense, formal discussions between two men in indoor environments. |
| 10 | Close-up portrayals of a woman's tearful emotional distress. |
| 10 | Suspenseful scenes of women in shadowy thresholds. |
They are pretty niche but I was happy to see “Fearful or suspenseful women's faces and eyes dramatically obscured…” as the number one cluster since that is the recurring visual trope that triggered this idea in the first place.
I built a cluster browser and “cleaned” up as much explicit material as I could to make it a public interactive tool. Of course I’m sure I missed something so if you don’t want to potentially see scary/gross/naked things don't use the tool below:
Of course if we are dealing with vision language models there are going to be problems. I noticed that sometimes I would rarely get a frame with a completely wrong description. I found that the VLM would generate a predictable description for some frames especially if it was an abstract or completely black image. It would just invent a realistic sounding description like “a masked figure lurks in the shadows. the woman's eyes widen in terror. blood stains the pristine white dress.” It would reuse the same duplicate wrong descriptions for many images so I could aggregate on them and find them. If you want examples you can see in this tool (loads 45MB of data) [https://thisismattmiller.github.io/giallo/apps/dupe-descriptions/]. The takeaway is that if you are using these models for “serious” work you better have a seriously good QA workflow process because it will generate bad data eventually.
I wanted to see if you could plot these clusters within the playtime of the movies. I was thinking maybe you could see that specific types of scenes happen a predictable times in these types of movies. For example something like, you often see the silhouetted killer creeping up the staircase at the half-way mark of the movie. I built a tool to plot the occurrences of the cluster members on a 0-100% x-axis. Technically it works, for example here is the plot for the cluster: “Cluster 3 - Opening credit sequences featuring airplanes and production details.”
You can use this tool to explore, plot and compare the other clusters:
If you explore there are interesting patterns you can extrapolate, for example: “Cluster 120 - Close-up depictions of men with bloodied faces exhibiting intense pain and distress.”
Which makes sense in the context of these movies often the male antagonist gets their comeuppance or justice for their crimes in the last act of the film.
That wraps up this fun exploration. I think if I could get my hands on more Giallo films from the 140 list I could do more, I would even likely increase the frame extraction to one every second. These VLM models come out quickly and are rapidly improving but as I’ve shown, care must be taken because it will generate bad data. Otherwise this seems like a potential approach to “distance read” a large corpus of thematically related films.