Asked as someone learning Japanese. Findings drawn from Japanese-specific research are framed for Japanese learners and may not hold for other languages; where the evidence is general second-language research, the report says so.
There is no source that tests English-speaking N4 Japanese learners watching Japanese TV specifically, so the answer has to be pieced together from general second-language subtitle/captioning research (mostly on other language pairs) plus one anime-viewing study with English speakers learning Japanese informally. The overall pattern from that broader evidence is that no-subtitles input is the hardest to learn from and gives the least measurable gain unless comprehension is already fairly high; Japanese (target-language) subtitles seem to help learners notice and connect sounds to words, which is the closest match to "listening ability acquisition"; and English (native-language) subtitles boost overall comprehension and enjoyment but let learners lean on reading rather than actually training their ear. For someone at N4, a level where vocabulary and grammar are still limited, this suggests Japanese subtitles or a no-subtitle/Japanese-subtitle mix are more likely to build listening skill than English subtitles, though none of the sources measured this directly for Japanese-English learners at N4.
None of the available sources ran an experiment on English-speaking N4-level Japanese learners comparing no-subtitles, Japanese-subtitles, and English-subtitles television viewing. The closest evidence is a study of English-speaking anime viewers with no formal Japanese training, and several subtitle studies done with other language pairs (mostly English, Spanish, and French as the target language).3,1,2,4
When on-screen text matches the language being learned (called captions, as opposed to subtitles in the viewer's own language), it tends to help learners recognize written words and connect them to sounds, while text in the viewer's native language tends to help overall story comprehension more.4
In plain terms
A review of nine studies on subtitled video found that beginner-level learners tend to rely more on subtitles in their own language to understand what's happening, and this group outperformed a same-language-caption group on comprehension tests.4
In a study of learners at a range of proficiency levels watching a TV series in three conditions (no on-screen text, target-language captions, or specially highlighted target-language captions), the type of on-screen text did not significantly change how much students felt they were learning, though vocabulary and expressions were rated as best learned overall regardless of condition.1
A study of native English speakers who had never formally studied Japanese but regularly watched subtitled anime found that 38% could correctly answer at least half the questions on a standardized Japanese listening test, and watching also produced measurable vocabulary gains, though more years of viewing didn't lead to better listening scores.3
In a large study of English-language TV viewing by foreign-language learners, having captions available improved comprehension mainly for the earliest episodes watched and for episodes that were more difficult, suggesting captions act as a support that matters most when the material is hardest to follow.6,7
Across several studies, listeners' existing vocabulary knowledge and their ability to break spoken sentences into individual words are strongly linked to how well they understand spoken language, meaning a learner's listening gains from any viewing method partly depend on how much vocabulary they already have.9,5,8
In plain terms
One study found that simply watching the same video twice produced comprehension and vocabulary gains just as good as using an interactive app that let learners pause, replay, and slow down the video — suggesting the number of exposures matters more than fancy playback controls.4