Bir YouTube Yorumu Kimin Hakkında? Havayolu Videolarında Hedefli Duygu AnaliziWho Is a YouTube Comment About? Target-Aware Sentiment Analysis on Airline Videos

Sentiment analysis usually asks one question: is the comment positive or negative? In this study I added a second one: who is the comment about? I labelled 6,659 Turkish comments under YouTube videos about THY (Turkish Airlines), Pegasus and AJet by sentiment, target and topic. The second question changed the answer to the first.

Data

I collected the comments from YouTube with a Python script I wrote. For each brand I used the same five neutral search patterns (such as “… flight experience” and “… review”). Searching with words like “delay” or “complaint” would have pulled the result towards negative from the start. I kept videos whose title names the brand and took up to the 300 newest comments from each. I did not collect commenters' names.

27% of the comments were not in Turkish; I identified the language with an automatic language detector. I removed replies, because a reply is written to another commenter and cannot be read correctly without the comment above it.

The videos fall into four types

The searches returned 115 videos. I assigned each one to a type by hand from its title; 8 were unrelated to the brand (a song with the same name, a match played by the airline's volleyball team) and were removed.

The distribution is uneven: 73% of THY's comments sit under ad films and 48% of Pegasus's under crash stories. I did not create this imbalance; it is how the search results came back. The most important finding below comes from it.

Labelling

Each comment received three labels:

• Target: Is the comment about the brand, about the video itself (the ad, the music, the narrator), or about something else (politics, prayers and condolences, other companies, chat)?
• Sentiment: Is the writer's attitude towards that target positive, negative or neutral?
• Topic: If the target is the brand, which topic? Safety, management, price, operations, service, comfort or general.

The labels were assigned by a language model (Claude). The model read each comment together with the title and type of the video it was written under and decided according to a written guideline. No word lists or rules were used; that is what lets a comment like “great, a three-hour delay again” count as negative.

To see how far the labels can be trusted, I had a random 400 comments labelled again in a separate pass that did not see the first labels.

Who is the comment about?

33% of the comments are about the brand, 35% about the video itself and 32% about something else.

Under ad films, 44% of the comments talk about the ad: its music, its actors, the feeling it evokes. Under crash stories, 45% are about neither the brand nor the video: prayers, condolences, general aviation talk. The brand is discussed most under news and analysis videos (55%).

The target also shapes the sentiment:

65% of the comments about the video are positive, while 50% of those about the brand are negative. An analysis that does not separate the two counts “what a beautiful ad” as praise for the brand.

What happens when the target is ignored?

Computed without the target, the negative share comes out low for all three brands: 20% instead of 36% for THY, 38% instead of 60% for Pegasus. The gap is largest for THY, because most of its comments sit under ad films and praise the ad.

Brand or video type?

Looking at the chart above, it is easy to say “THY draws far less criticism than Pegasus”. First it is worth seeing how the negative share changes with the type of video.

Same brands, same platform; the negative share is 26% under ad films and 83% under crash stories. Splitting each type by brand changes the picture:

Under ad films THY is at 21% and Pegasus at 23%: no difference. Under news videos they are at 53% and 63%; the intervals are wide and overlap. The overall gap between the two brands depends on which types of video their comments come from. To put that into one number, I reweighted Pegasus's shares to THY's mix of video types:

• The gap between Pegasus and THY comes largely from the sample. If Pegasus's comments had the same mix of video types as THY's, its negative share would be 39% rather than 60%; THY is at 37% on the same types. The interval is wide (34–59), so a real difference cannot be ruled out; what can be said is that the raw gap is not evidence of one.
• For AJet the gap does not close. Even under its own official videos, 87% of the comments about the brand are negative. But that figure rests on 54 comments under 4 videos, and 55% of AJet's brand comments come from just three videos.

What are the negative comments about?

• Pegasus: 45% of the negative comments are about safety, and 85% of those were written under crash videos.
• THY: The negatives split in two: safety (38%, videos retelling old crashes) and management (36%, sponsorship deals and how the company is run). Meanwhile 52% of all comments about the brand name no topic and are mostly pride and praise.
• AJet: The largest share is management (27%), mostly about the rebranding and the company's future. Operations and service complaints take a larger share than for the other two.

Price, operations, service and comfort, the passenger's everyday experience, make up only 23% of the comments about the brand. YouTube comments are a good source for brand perception and the news agenda, not for customer-experience complaints.

Can a cheap model do the same job?

Labelling 6,659 comments with a language model is feasible. With hundreds of thousands of comments every week it gets expensive. So I used the language model's labels as a teacher and trained a small model: TF-IDF on word and character n-grams with logistic regression on top. Each time, the model was tested on comments from videos it had never seen.

• Counting words is not enough. The lexicon approach agrees with the language model on only 51% of the comments and misses negatives: it puts the negative share of THY's comments at 10%, against 20% in the labels.
• The small model stops at 71%. It disagrees with the language model on three comments out of ten. That is not enough for work that looks at individual comments.
• A random split is optimistic here too. With comments from the same video in both training and test, accuracy rises to 74%.

In practice one usually tracks a share, not individual comments. I computed the negative share of comments about the brand from the small model's predictions and compared it with the labels:

A model that disagrees with the language model on 29% of the comments gets the brand-level share within 1–5 points; the errors largely cancel out. For tracking a weekly indicator the small model is the right tool, for classifying a single comment the language model is.

How many labels are enough?

With a thousand labels the model learns most of what it will learn; after that the gain is slow but still there. Since the curve has not flattened, more labels would raise comment-level accuracy a little further.

Second pass: measuring the labels against a human

The weakest point of the sections above was this: the labels came from a language model and were never compared with human labels. To test that, I redid the study from scratch with a different pipeline. The cleaning rules changed (replies and brand-unrelated videos were kept this time; spam, bots and the brands' own replies were removed instead), so the analysed set is 8,889 comments rather than 6,659. Sentiment came from an open-source Turkish BERT model (savasy/bert-base-turkish-sentiment-cased) instead of a language model; target and topic labels came from transparent rules.

I then stratified 120 comments by brand and target, read them one by one and labelled them by hand. The model was measured against those labels:

Accuracy 64.2%, Cohen's κ 0.455. The target labels do better: accuracy 78.3%, κ 0.675 (about the video F1 0.84, about the brand 0.79, other 0.72).

Where the errors gather says more than the accuracy figure alone:

• The problem is not telling sentiment apart, it is saying "no sentiment". Leaving the neutrals out and looking only at the 67 genuinely positive or negative comments, accuracy rises to 88.1%. Polarity is strong, neutrality detection is weak — what you would expect from a binary-trained model.
• κ 0.455 is a modest figure. The net-tone numbers carry a few points of uncertainty. Comparisons between brands hold; absolute values do not.
• Two pipelines reach the same headline. The first pass found 33% of comments to be about the brand; the second, with entirely different cleaning and labelling, finds 30.4% (2,706 of 8,889). The page's main finding is not an artefact of one method.

What this validation does not cover: what was measured is the second pipeline's BERT + rule labels. The language-model labels in the sections above are still not compared with human labels; that dataset is no longer available to me. And I assigned the 120 labels alone — with no independent second annotator, inter-annotator agreement is unmeasured.

Was the comment written on the brand's own channel?

The second pass asked a question the first did not: was a brand-targeted comment written under a video on the brand's own channel, or on an independent one? Same brands, same period, same model — the only variable is who owns the video.

A 49-point gap. And it feeds straight into the brand comparison: 52.6% of brand-targeted THY comments come from THY's own channels, against 12.4% for Pegasus and 11.9% for AJet.

In the raw data THY leads comfortably at +20.1. Narrowing to brand-targeted comments pulls all three down, and much of the remaining gap is explained by where you measured. This is the page's second lesson in measured form: in reputation reporting, owned media should be reported separately, and the single number that reaches management should come from independent channels.

Limitations

Conclusion

A comment written under a brand's video does not have to be about that brand. In this data only one in three was; counting the rest made the brand look more positive than it was.

The second lesson is about sampling. Most of the 23-point gap between THY and Pegasus came from which types of video were collected. Before comparing a sentiment score between two brands or two periods, it is worth checking whether the mix of sources is the same.

A third lesson came out of the second pass: whose channel the comment was written on moves brand-targeted net tone by 49 points. The first question to ask of any reputation metric is which channel the number came from.

Data: YouTube comments, collected on October 7, 2026. Comment texts and author details are not shared. The shares describe this sample; they are not an assessment of the companies' service quality.