Site navigation

Response from the Open & Closed Project

This intervention is the response from the Open & Closed Project to the claims made in CRTC 2008-8.

The Open & Closed Project is an independent research project committed to researching a set of standards for captioning, audio description, subtitling, and dubbing; testing them in the real world to prove they work; publishing the standards; and training and certifying practitioners. Those activities would be carried out in the open; anyone could contribute.

The CRTC and Canadian broadcasters have refused to fund the Project. At $4 million to $7 million with a first-year cost of $500,000, cost clearly is not the object. CRTC and broadcasters have shown they are actively hostile to genuine scientific research carried out in an open process.

CRTC suborned scientific fraud

The CRTC suborned scientific fraud two ways:

Inviting broadcasters, one captioning firm, and lobby groups to convene in secret to write a claimed standard
Standards are properly developed by a wide range of parties, including qualified scientists and researchers. While occasional documents or meetings may be legally privileged (e.g., due to patent concerns), standardization processes take place in the open so they can be scrutinized. The current process is invitation-only and secret.
Inviting – by phone call – an industry lobby group to write a report with a foregone conclusion
The CRTC asked the Canadian Association of Broadcasters to engage in an “exercise” of “verification” of the claimed standard developed by a secret, invitation-only group.

CAB did not engage in actual scientific fraud because it chose a market-research approach. Even that did not produce reliable data.

The “validation exercise” did not even begin to simulate real-world television viewing. Subjects sat down in a well-lit training facility before computer monitors that showed captioning simulations, not real captioning. The Project has not encountered so deceptive a research arrangement since the research into the Tiresias typeface, which tested television captioning by showing printouts on a monitor.

Rejection of predetermined outcome

The purpose of the “validation exercise” was to prove that scrollup captioning is accepted on all forms of programming. Respondents rejected that claim.

  1. While the summary admits that one-third of respondents want scrollup captioning banned, it also claims that two-thirds of respondents would tolerate scrollup captioning. But the only distinction of interest is preferences on the programming styles where scrollup is known to fail. Scrollup captioning was rejected completely by the English group, and by 83.3% of the French group, for “drama/comedy.” Over 95% of English respondents and 75% of French respondents rejected scrollup for movies. The report falsely claims the data “impl[y] that two-thirds [of respondents] do not have an issue with scrollup captioning for at least some programming.” The use of scrollup captioning for some programming styles is noncontroversial; the report leaves the false impression that respondents accept scrollup on all programming. The opposite is the case.
  2. All programs cited by respondents as their favourites are captioned in pop-on.
  3. The report asked a leading, closed-ended question. Instead of asking for respondents to state their preferred captioning style, respondents were asked a yes/no question: Do you fail to have a “specifically preferred captioning style”? The report implies that the results (8.7% English, 25% French) amount to respondents’ endorsement of all-scrollup captioning.
  4. The report asked for Likert-scale responses to the statement “Captions should be synchronized with the image on the screen.” All respondents found that statement somewhat or very important. But only pop-on captions are synchronized.

Researchers at no time asked a question that honestly reflected their intentions, e.g. “Broadcasters would like to save money and time by using scrollup captioning on any programming they wish, including drama/comedy, movies, and potentially all programming. Do you agree or disagree with this idea?”

The only statement in the report supported by the report’s own data actually contradicts its foregone conclusion: “Pop-on captioning is preferred for drama/comedy, movies, children’s programming, and documentaries.”

There were other research deficiencies that corrupted the results.

  1. Unrelated questions were asked, often on the basis of technical impossibilities (e.g., changing the font in Line 21 captioning).
  2. The procedure ignored even the most basic research methods, like counterbalancing the order of stimuli.
  3. The French video sample was almost twice as long as the English.

This is not a standard

It is an offence to scientific propriety to call the CAB document a standard. It wasn’t developed in an actual standardization process and does not function as a standard.

  1. It fails to explain what to do and what not to do in captioning, and why, with research justification.
  2. It fails to cite any research.
  3. It hasn’t been peer-reviewed.
  4. It hasn’t been tested.
  5. It doesn’t take into account the multiplicity of formats on television. How does one caption a subtitled program; an operatic performance; a musical in which only dialogue, not songs, is dubbed; a program with two narrators; an open-described program; language-learning programs; spelling bees and dictée; commercials shorter than 30 seconds; music videos; filmed stageplays; sign-language video? How does one supply two caption streams, e.g., verbatim and simplified?
  6. It included not a single image or video clip.
  7. No training or certification methods were discussed. Standards are not standards without consistency across the board, something that can be achieved only with training and certification.
  8. It works from the outdated pretext that captioning takes place using videotape and post-facto encoding.

The claimed standard was presented unfinished. Here is a verbatim extract from p. 17:

  • A pop-on caption normally appears in a pyramid or inverted pyramid shape. The text is generally centred.
  • Two-line captions are the norm and are preferred over long, one-line captions.
  • One-line captions are recommended only when a sentence is very short:

Where is the example from the third statement?

An existing captioner could not use the document to upgrade or alter their methods to meet the claimed standard. A person new to captioning would not be able to use the claimed standard as a learning guide.

Firsthand report

A subject who attended one of the sessions described the scene:

They had 24 volunteers and two rooms – one for deaf and one for hard-of-hearing. We were crammed into these small rooms and sat at several rows of desks to watch the samples on 12″ computer monitors. Watching videos will full-sized captions from two feet away is not ideal or even realistic.

There were four variations of the pop-on/rollup captions, but the questionnaire was limited to generalities (i.e. “How important is it that captioning be legible?” and “What are some positives and negatives of rollup/pop-on?”).

Every variation of the captioning had some problem with it: One style used a thin-weight condensed typeface, another had the opacity at 60% or so, and the descenders on one of the typefaces were cut off by the black bar. But they did not invite us to address any of the individual styles or even discuss the variations.…

The DVD cycled through four types of programming: movie, prerecorded comedy and drama, soap opera, and children’s. No sports, no newscast, no talk shows or live casts – nothing with a crawl or information on-screen (aside from a couple of intro credits), no overlapping dialogue. And no commercials to play after the delayed captioning to show how much information actually gets lost when the feed switches.… Most of the volunteers were elderly and just grateful that the captioning wasn’t coloured.

“Universal design” is a myth

The Alliance for the Equality of Blind Canadians and the Centre for Learning Technologies (CLT) claimed that the entire apparatus of accessibility in Canada is flawed because it does not comply with their preferred ideology of “universal design.” Neither of these parties recognizes that universal design is a myth. So-called universal-designed products are always products custom-designed for people with disabilities.

As Bringolf (2008) explains:

Clearly universal design cannot be understood without reference to disability. This is not an issue in itself: Universal design automatically includes people with a disability, but the semantic difference is that it is not specifically for people with a disability, thereby suggesting the exclusion of others.

Once locked into the disability scenario, the knock-on effect is that designer thinking defaults to disability-discrimination legislation, accompanied by fears of litigation.… As we can see, the domino effect of using one term to mean another has the power to fracture and distort a concept whether intentional or not. It has the power to remove universal design from the domain of being for everyone to that of being disability-specific.…

[In] the results of two research projects that focussed on consumers and their preparation, or lack thereof, for aging lifestyles… “[u]niversal design” failed to register with consumers who had little, if any, idea of what it meant.… A product, therefore, labelled as a “disability” product has no appeal, even to people with a disability. The conclusions drawn were that the term “universal design” should be abandoned because it will have no appeal to consumers (or designers) regardless of how efficacious it is proven to be.

Clearly those who wish to continue the promotion of universal design in its original form are in a fix. Lack of understanding and misusing “universal design” has created a void in which “accessibility” and “disability” now reside. As such, it has evolved from a process to a product; a disability product. This was unintentional, but we cannot turn back time. Universal design is a synonym for “disabled” design in the hearts and minds of disability rights activists, legislators and designers alike.

Universal design is a misnomer when dealing with television broadcasting. According to the ideology, the creator of a program must take disabled people into account at the outset. Who is the “creator” of a newscast? an operatic performance? a music video? five episodes of a television series? How can these creators, unschooled as they are in fields like captioning and audio description, be expected to have any skill at all in captioning and description?

So-called universal design places an onerous burden on writers, directors, and others who just want to write, direct, or carry out their chosen professions. Taken to its logical end, creators would be forced by law to do work they don’t want to do and aren’t qualified for. In addition, prerecorded programming could never be made accessible because we would have missed the chance to design universally at the outset.

Theoretical research is not useful in this context

CLT is engaged in a long-term process of securing government funding to carry out theoretical research in accessibility. The organization starts from the assumption that current captioning and description are, at root, failures that need replacement.

Theoretical research is necessary in every field. It deserves significant funding support and rigorous peer review. But CLT’s own results show that its proposed methods are unpopular with real-world users. Real-world captioning and audio description require applied research, not theoretical research.

CLT’s program of research is of little or no use in the current CRTC proceeding and does little or nothing to improve day-to-day captioning and description. Additionally, CLT tends toward poor choice of research partners. CLT will audio-describe film Blindness in postproduction and call it a universal-design exercise. This would be of interest were it not for the fact that Blindness was the target of serious protests, including on-street demonstration, by actual blind people.

CLT’s research seems attractive to CRTC and CAB – someone else pays for it, it has no bearing on the real world, and would not affect what the CRTC and CAB intended to do all along. While the CRTC and broadcasters may entertain the notion of showering funding on CLT to take over the work the Open & Closed Project wishes to do, CLT is incapable of applied research and applied researchers won’t work with CLT.

Funding

The Open & Closed Project could easily be funded by an industry deregulation plan that traded near-complete or full deregulation of non-broadcast channels for full accessibility of the entire broadcasting system. The Commission has received a petition to consider such a plan, which the Project reiterates and supports.

Number of sign-language users in Canada

The Canadian Association of the Deaf claims “no fully credible census of [deaf people]” has ever been carried out. CAD provides this as an answer to the question “How many deaf sign-language users are there in Canada?” although it does not actually answer the question.

Statistics Canada’s 2006 Participation and Activity Limitation Survey (PALS) documented:

While CAD has objected to Statistics Canada’s methods in the past, the current PALS seems to have met many of its objections. It is, in any rate, the only reliable set of estimates. CAD is not staffed by statisticians and has never provided its own set of credible statistics, other than reiterating a catchphrase of a “traditional ‘one in ten’ formula” that has no basis.

If you’d funded us, we’d be done by now

The Open & Closed Project has sought funding for five years. We projected only a four-year duration to finish our first standards. Hence, had the CRTC and broadcasters invested in the Project, by now we’d already have a genuine standard available for a year’s real-world testing and we wouldn’t be having this discussion.

Reference

Bringolf, Jane (2008). “Universal design: Is it accessible?” Multi: The RIT Journal of Plurality and Diversity in Design 1:1 (PDF)

POSTED: 2009.01.12 13:42