Filmmaker

Just noticed something peculiar, whilst reading a review of THE SIEGE on DVDTalk--reviewer Aaron Beierle gave these review stats to the original non-anamorphic release:

MOVIE: 2 stars
VIDEO: 5 stars
AUDIO: 4 stars
EXTRAS: 1 star
REPLAY VALUE: 3 stars

Fair enough, but then I took a look at his more recent review for the new 16 X 9 anamorphic, DTS-encoded DVD of THE SIEGE, which gets the following stats:

MOVIE: 2.5 stars (however, it's the exact same movie--not a director's cut or such)
VIDEO: 3.5 stars (he explains that the anamorphic image is a substantial improvement over the original release, which earned 5 stars!!!)
AUDIO: 3.5 stars (he says the audio is at least as good as the original release and it, of course, now features a new DTS mix--but somehow, we've dropped half a point?)
EXTRAS: 0.5 stars (the new DVD contains the same sole feature as the old DVD--the original trailer--but again, we've inexplicably lost a half point)
REPLAY VALUE: 1.5 stars (same movie, better presentation, but the replay value is cut in half?!)

Now, this is not meant to disparage Aaron, who I don't know from Adam, but to point out the sad fact that, in the rush to become the most saturated DVD review site on the Web, DVDTalk is chock-a-block with reviews that lack consistency and a certain degree of accuracy. It is unfortunate--I really love DVDTalk's forum feature, but the review section continues to prove to be of almost no use to me...

tommy28

You have WAYYY to much time on your hands,

tommy

Adam Tyner

Often opinions change on viewing a film for the second or third time, and personal standards change over time. What constituted 3 stars for Aaron in 1999 (arbitrary year) doesn't necessarily hold true today.

As a DVD Talk reviewer from the beginning as well as the programmer who wrote the current reviewing software, I assure you that saturation is not the goal. Bear in mind that Aaron writes reviews for his own site and shares them with DVD Talk, so he's in a very different class than the rest of us, who submit reviews at a very different pace. I've only written around 15% as many reviews as Aaron in the same time period, for instance.

I definitely agree that inconsistency is a problem with the reviews though, and I started a thread about that here: http://dvdtalk.com/forum/showthread....hreadid=115314

Filmmaker

Originally posted by ctyner:
Often opinions change on viewing a film for the second or third time, and personal standards change over time. What constituted 3 stars for Aaron in 1999 (arbitrary year) doesn't necessarily hold true today.

Understood, and it (partially) gets Aaron and his ilk off the hook for variation in the MOVIE rating, but is there something in the review software that would prevent him from editing the technical ratings in his original review to reflect something more consistent with the new review? As it stands now, I don't know which review to take seriously or which one to perceive as accurate and useful for purchasing/renting purposes.

Adam Tyner

The more current the review, the more likely that it better represents what the reviewer feels. And yup, the software I wrote allows reviewers to go back and update their reviews as time goes on.

Also, if you want to contact a reviewer about something he or she wrote, click on the reviewer's name in that review, and it'll take you directly to a form where you can compose an e-mail.

Filmmaker

Good enough, I suppose.

I appreciate your concern and responses, ctyner.

Liquid Death

I agree with what Adam says.

The Siege (Non DTS) came out April 20th, 1999.
The Siege (DTS) came out in December of 2000 (if I remember correctly).

Thats a year and half difference - certainly, what constitues a 5 rating changes over time, and one's views on a movie changes over time as well. Aaron has viewed WAAAY more DVDs than I have, so I feel confident that his scores are justified.

DVDRules1

I'll reinforce what everyone else has stated.

If you look at when the reviews are posted, they were actually posted exactly one year after the other.

Certainly, feelings change about what makes great software and the movie itself in 12 months. I'd probably reviewed a few hundred titles in the span of time between writing the reviews in question.

After loading as many reviews as I have onto DVD Talk, I'd guess that I simply didn't remember to even look to see what I'd said about the original release since it was a year prior. Having another site of my own, I do not have a list of what I've shared with DVD Talk and have not - I often find reviews that I'd thought I'd uploaded and haven't.

G. Noel Gross

I printed up some sheets that I record the various ratings on ... that way when I review a new disc, it's a snap to compare it to the other ratings I've given in the past. Makes it a lot easier for me, and hopefully more consistent.

Filmmaker

Good for you, noelgross--frankly, I'm a little distraught at the cavalier attitudes from Liquid Death and DVDRules1. I realize that there's a year's difference between the two reviews, and that standards might vary in that time, but I refuse to accept that it is that difficult or inessential to simply go back a re-rate the original review accordingly. If a review is expected to potentally be obsolete after a year or so, why waste valuable bandwidth keeping it around?

Liquid Death

I don't think my attitude is "cavalier," and in fact, my statement was that what constitues a particular score changes over time. You can't review titles based on what WILL come out, only what HAS come out. Therefore, a film which gets a 5 in the extras, for example, might rate a 4 later on when more loaded SEs come out (certainly as we're seeing now). Does that mean reviewers need to go back and update every review they've ever written to keep things consistent?

I don't think so, as that would be a continual thing, and I just don't have the time to do it. I believe Adam is putting dates on the reviews (when the review was written), and that would be a good way for people to put the review in perspective.

Adam Tyner

Yup, this version of the review software prints the date of the review now.

Gil Jawetz

The way I see it, older reviews are being archived, not maintained, and they shouldn't really be changed much. In the few instances where I've HAD to go change an old review (like when the Any Given Sunday 2 disc came out on its own after I reviewed the box-set only version) I made a note that the review had been changed. What was a 5 star transfer 2 years ago reflects the standards of the time. If the standards change then that's just the way it is. Gone With The Wind is said to have made $77 million, but if you took inflation into account it would have made WAY more than any other movie.

Anyway, this is why there is content in the reviews. The number of stars shouldn't tell the whole story.

Filmmaker

Sorry, I still find the attitude cavalier--if you don't have the time or inclination to keep your reviews viable, why should I depend on them for making my buying and renting decisions? Also, let's keep in mind here that we're talking about reviews for the same title in multiple editions. Typically, that's a relatively minor number of titles to do "upkeep" on. If updating the old review is such an issue, why don't you simply delete it (since it is inarguably wasted space) and add a line to the new review stating something along the lines of "forget the old version--this is the one to get!" or, even better, adding a few lines of comparison and contrast? As it is, there are a fair number of useless reviews in the database that, when compared to a newer release of a same title, tend to raise more questions than answers. You ask me to assume that the new review is the more accurate. Well, you know what they say about assuming... Am I to look at the old review and assume that the reviewer saw more flaws in the film and/or presentation the second time around, or should I perceive that the reviewer lost track of the elements that he found so enjoyable about the film/presentation the first time he saw it? I really don't mean to belabor the point, and I recognize the effort that likely goes into even the most ineffectual review, but I am letting you know that at least one member of your potential reading base, who would love nothing more than to get use out of said reviews, is finding the bulk of DVDTalk's reviews to be amateurish, inaccurate, inconsistent and ultimately, immaterial. I doubt this is effect Geoff and the rest of the review gang are shooting for, but it's what I perceive to be the truth. You guys have a choice--recognize some of your failings as reviewers or keep trying to tell me why I'm mistaken. One of the two just might increase readership and the validity of your efforts...

Adam Tyner

Could you elaborate on some of the other problems? I definitely agree that there are problems, many of which I commented on in the Forum Feedback thread I mentioned earlier, but I'd like to hear more from someone who's not a reviewer... Be as specific and detailed as you can, unless you really hate my reviews.

rasalas

I clearly see Filmmaker's points. As far as Tommy28's remark that Filmmaker has too much time on his hands, if you look up The Siege using the review search engine, three entries come up--two by Aaron. It doesn't take a lot of effort to read them and see the disparity.

I think the biggest problem here is not so much the difference in ratings but the reviewer's lack of acknowledgement of it. All it would take is a statement in the new review saying that his opinion has changed since the original release or that his grades are tougher--anything to signal that he knows what he wrote in the past. Unfortunately, the current impression is that he doesn't.

Liquid Death

I must have misunderstood something, as I figured you meant reviews in general (and not updated edition DVDs), in which case, I wouldn't have the time to update all my reviews every Tuesday to reflect new standards in picture/sound/extras as compared to everything else I've previously written. That is what I was previously trying to say.

Knowing that you're now only talking about updating reviews of previous versions of a film when a newer version comes out, is of course, a different matter entirely. I agree that if someone reviews an updated version of something and HAS seen the original, then a few notes of compare/contrast would be nice. I haven't been in that spot yet (tho I did review the original error version of A Better Tomorrow and then posted a note saying the the soundtrack was in error when I found out).

Liquid Death

If you could possibily give a few examples of what you mean in particular, The Siege review aside, then that might help. As Adam stated above, some stuff was covered in the thread he posted, such as grammar mistakes.

GeoffK

Not liking the tone of all this. Think it's time to close it.