1.4 Critically Assessing Media Content

Navigating and sorting through the abundance of information can be challenging. When on social media, content from family and friends, media and companies, experts, influencers and more is mixed together. Awareness, critical assessment and evaluation of what the user see become aspects of media literacy that need to be applied constantly.

Algorithms to customise the content we see on social media and in search engines have significant influence over the information that each individual user is exposed to. While on one hand provide content that is deemed relevant to the user’s target group, previous use and perceived preferences, algorithms can also be used to help spread disinformation.

With some knowledge or awareness about how algorithms are used, it becomes easier to evaluate or critically scrutinise the search result and question why something appears in the social media feed, web search or the online news page.

Tests to Assess Search Results and News

To answer three of the questions in this section, the respondents were shown images of online search results and realistic news sites where they had to identify different elements. The images were screen shots of online search results, one for a down jacket and one simple search on the current prime minister's name. The third question showed a mock-up screen shot of a major online news site. The images were separate for each country, in each country's language. Different elements of the images had reference numbers to connect to the survey question. For reason of space, examples of the images can be found in Appendix III Data.

How is the online search results listed?

In the image of an online search for a down jacket, a majority of the respondents correctly stated that the order of search results is determined by someone having paid to have their jacket listed at the top. The largest share of respondents to believe this were found in Finland and Iceland. However, there is still about one in ten of the respondents in Sweden, Norway and Denmark who believe the jacket listed on top is “the best”. When adding the result from two of the other options, listed at random or being unsure about the order of search results (don’t know- option), the share of respondents is about two of ten in the same countries.

Figure 1.4.1. Evaluating search results (down jacket) 16+ years by country (percent)

Identifying source of search results

In the test to categorise the result of a search on the current prime minister's name items are mostly correctly identified. Especially the results of social media content were identified, 50 to 70 percent of the respondents in each country made a correct categorisation. The same with news content and search hits from an encyclopaedia (Wikipedia). There are some uncertainties about how to classify search results from a political party's website or search result from the government or parliament, but many correctly categorise them under 'other content'. To be noted here is that the reply option 'don’t know' is often relatively high, which indicates that assessing search results seems to have caused some difficulties for respondents. This may be something to consider in media literacy training.

The Swedish result is included below as an example. The correct category is listed in the column to the left.

Figure 1.4.2. Categorising search results (prime minister), 16+ years Sweden (percent)

Assessing news media content

Editorial and commercial content is generally correctly identified in the mock-up news websites in all countries. However, it is not clear-cut for everyone what type of content the different sections of the newspaper website represent (news or other editorial content). Also, in this question, the reply option 'don’t know' is often relatively high.

The Finnish result is included below as an example. The correct category is listed in the column to the left.

Figure 1.4.3. Categorising news media content 16+ years Finland (percent)

Key insights

Paid ranking of search results is recognized by a majority, but significant portions of the population lack awareness of such mechanisms. In a test regarding ranking of online search results, most participants correctly identified that paid placements end up at the top of search results. Still, many respondents lack an understanding of search engine mechanisms. About one in four answered either that they believe that the top search result is to be considered as the best, that it is randomized, or that they don’t know.
Identifying sources or who posted content listed in an online search result is mostly correctly identified. However, some categories seem to have raised more uncertainty, and the share of respondents indicating ‘don’t know’ is above 10 percent. Being aware of the source or sender is important to be able to assess the value of the information.
Discerning editorial from commercial content on news sites display uncertainties among respondents. Whether content is editorial or commercial in an online news site will have impact on how it will be assessed.