• Jason de Puit

    This was really insightful. I particularly like the last point. If you love the look of your bike and your gear you’ll enjoy riding more often! An emotional response can certainly have more impact than ensuring your chain matches the rest of your groupset, for example.

    • echidna_sg

      slippery slope… imagine how much more you would love your bike if the chain DID match though? ;-)

      • Shirley Walter

        my associate’s stride close relative makes $98 an hour on the portable workstation……….Afterg an average of 19952 Dollars monthly,I’m finally getting 97 Dollars an hour,just working 4-5 hours daily online.….. Weekly paycheck… Bonus opportunities…earn upto $16k to $19k /a month… Just few hours of your free time, any kind of computer, elementary understanding of web and stable connection is what is required…….HERE I STARTED…look over here
        ???? http://GoogleGoodPayOnlineJobsNetworkOnlineCenters/$98hourlywork…. ??????????????????????????????????????????????

      • Maria Blackburn

        my associate’s stride close aunty makes $98 an hour on the portable workstation……….Afterg an average of 19952 Dollars monthly,I’m finally getting 97 Dollars an hour,just working 4-5 hours daily online.….. Weekly paycheck… Bonus opportunities…earn upto $16k to $19k /a month… Just few hours of your free time, any kind of computer, elementary understanding of web and stable connection is what is required…….HERE I STARTED…look over here
        ???? http://GooglePrimeJobsOnlineProfitableJobsNetworkOnlineCenters/$98hourlywork…. ??????????????????????????????????????????????s

  • Ritch

    How about a rating out of ten for each of “Strong”, “Light” and “Cheap”. No one will be able to accuse you of panglossian reviews if you score them according to the first metarule of cycling equipment.

    • echidna_sg

      great thought!
      I’d hate this job! I’d spend too much time editing my work and checking that I didn’t EVER say “laterally stiff and vertically compliant” or variants thereof! ;-)

  • jules

    nice post Matt. a while ago I used to be involved with road testing new cars. I formed a similar conclusion to your #6 – it’s pretty rare these days for high volume manufacturers to put rubbish out in the marketplace. I’d suggest partly this is due to much better access by consumers to information (the internet and product reviews, and customer reviews) and also globalisation of industry (gone are the days of ‘the Germans know how to build a car/bike’ – those Germans will happily sell their IP to Korea, Taiwan or vice versa as may be the case).

    • I have to admit, I’ve sent back products offered to us because right off the bat I knew they would get a horrible review. This leaves me conflicted however. I should really be letting you know what not to spend your money on, just as much as the other way around. However, I have better things to do than spending hours reviewing some under-performing product, and the dramas it causes sucks up more time (eg. going back to the brand with all your reasons and tests, them asking you to try future iterations, etc…). There is this element to product reviews that I’ve faced before. That said, I’m doing a review on the Apple Watch right now that’s not going to be so favourable…

      • jules

        Apple can wear it! in the automotive industry, there were some fringe products (not cars, but questionable accessory-type ‘additives’), the supplier’s business model for which included suing bad reviewers for defamation. they won or settled decent amounts too. their products were to informed people – rubbish and misleading, but.. the law is an ass. so basically I can understand not wanting to get involved with reviewing problematic products.

        • Steel

          That’s correct Jules. I know because I used to do the same job as you.

          But yeah, there are genuinely no real lemons any more. The closest would be Jeeps which trade purely on brand cache and marketing and not on quality. Some of the new market brands (China and India) also have a bit of catching up to do, but they’ll get there.

          With bikes, I’ve not owned anything that wasn’t fit for purpose and very high standard – did what it says on the package. Perhaps with the exception of my Avid Juicy 3s which were friggin awful.

          So with cars if you want something to go from A->B you can dartboard the cars pages and you’ll probably be ok. The rest is just a value judgement as to whether you really want to pay for those extra features or style.

          The same applies with bikes: nicer components or frame construction adds dollars. You work out whether you want to pay the extra asked by the maker.

          It all sound terribly boring, but this is what Jeremy Clarkson discovered some 20 years ago. Cars are boring because they all work and are globally homogenised. He found excitement in car reviews by turning it into entertainment with the consumer information side of it going out the window.

          • jules

            the exception would be sports cars, but the problem is you have to break the law to enjoy the difference. not that I ever did that..

      • Please include the dodgy stuff, to some extent at least.

        Matt’s point #6 makes sense to me. This means I generally don’t want reviews to answer ‘Is it good?’ I want to know ‘Is there a problem?’

        • James Hall

          Maybe have the secret reviewer? Different website hosted on a server in an independent country

      • Francis Tan

        i would actually be interested in those items that you return. I think theres a greater sense of responsibility to be able to tell people what not to buy compared to telling them what to buy don’t you think so? ive always wondered why you guys dont review certain items eg. Cervelo bikes.. is it bec you return them right away knowing they are rubbish?

        • jules

          one of the more insidious practices by product suppliers is to withhold test products from ‘un-friendly’ reviewers. the objective is to train the reviewer to be positive towards them. this only works for products with a high profile, where readers will miss them – and potentially switch over to publications that do review them.

          I’m not suggesting that’s the case for Cervelo at all – in fact, they look like damn nice bikes to me. but beware the reviewer who is always first to get the newest toys :)

          • Holby City

            Have you ever ridden an R5?

            • jules


          • Paulo

            Actually you could just list the “no-test” products. No explanation necessary, no review, no effort and nothing to dispute with the manufacturer. But potential buyers get a “buyer beware” heads up.
            That maybe to simple though.

        • We haven’t reviewed Cervelo simply because the distributor has never sent us anything. We’ve reached out but haven’t had any luck. It’s probably the same with most other items as well. On the flip side, brands like Trek, Giant, Scott, Specialized are excellent at keeping us informed of their new products and are sending stuff in all the time.

          • duckingtiger

            Hi Wade, just wondering how do Ctips select products for review. I think i have read you written somewhere that

            1. You reach out to distributor for products you think is interesting. (e.g. latest aero bikes/ wheels)
            2. Distributors send Ctips products and you choose what to review, but you will choose what to review or not.

            Do site sponsors have priority when you have say, 20 products to review sitting on the table?
            Just wondering how you guys go through the review process. You always disclose if xx brand is a sponsor in the review, something we don’t see very often these days.

            • It’s a good question. Our number one priority is thinking of our readers and what they’d like to read and know more about. Products like the new Garmin, a new flagship race bike, etc are always things that people are interested in, so we are proactive in reaching out to the brand managers.

              There are many brands who regularly send us stuff in for review. We have close relationships with many brands and we’re happy to accept their products for us to test. All things being equal interest, the advertisers who are paying to be here get priority in terms of being in the queue.

              We don’t review items that won’t be of great interest to our readers. We’d love to review everything, but unfortunately that takes lots of time (and costs me lots of money for very little to no benefit). We need to prioritise.

              I’ve told a few brands that the product they’ve sent in is so inadequate that I would recommend improving it, then sending back for review.

              • duckingtiger

                Thanks Wade, truly appreciate the openness.

              • Aaron Heaysman

                Would be interesting to know the brands that don’t send back an improved product, probably brands to stay clear of.

                • I’ve told a small Australian clothing brand in two instances that they heed to improve before we review. But if I said who they were, people would tell me that I’m harming a small Australian brand and depriving them of sales. Either way, we can’t win here.

          • De Mac

            Perhaps the foibles that many experience with the Cervelo BB is the limiting factor here….

          • Francis Tan

            Thanks. I am one of those few Cervelo fan boys :) and i like the format of the reviews here so Im just curious what CT would have to say to their bikes :)

        • Holby City

          Cervelo are doing just fine without reviews here but I would like to see a Cervelo on Bikes of the Bunch.

      • Michele

        Wade, you should’ve waited until the 2nd gen Watch and Watch OS2. :)
        I can review the watch for you now: It’s rubbish!

        • Roger That

          I don’t know if I’d say it’s rubbish. I really quite enjoy mine, but it’s not ‘there’ yet. As lots say, the ‘killer app’ that really makes it or defines the space the Apple Watch can operate in is yet to be on the scene. That said, I wouldn’t wear any other watch now – and I collect watches (cheaper than bikes)!

          • Spider

            Not by much…the vintage watch scene has exploded! we could set up a watch for bike swap meet!

      • Lyre_bird

        Same thing happens in the wine world with a slight twist: Name brand stuff is always drinkable but usually boring (and frequently overpriced). Smaller producers are much more variable but the reviewers simply don’t write about the bad ones.

        • It’s not that the reviewers are trying to hide anything by not reviewing bad products. It costs a lot of time (therefore money to pay the reviewer), and personally I’d rather publish a review about a worthwhile product than something that hasn’t been well thought out and is rubbish. Unless of course it’s a popular or interesting product. We just published a negative review about the Recon Jet for example.

          • Lyre_bird

            I didn’t mean to imply that anyone was hiding anything; the general feeling in the wine industry is that people want to read about the interesting and good wines and that small producers with dud wines will either get better or go broke.

            • Sorry – I think I responded to the wrong comment!

      • jakub

        the article begins with a statement how important is to have reviews unbiased, yet here you clearly admit to a selection bias ;). in any case, if you’re being endorsed by certain brands, your reviews can be hardly unbiased, no matter how hard you try. human mind is so prone to many unconscious biases, even in less obvious cases. that’s why from otherwise superb content I always skip your product reviews (and many other product reviews in general). on the matter of biases, from the popular stuff I recommend reading Predictably Irrational by Dan Ariely or Thinking Fast, Slow by Kahneman/Tversky. definitely a nice insight into how our brains and decisions are irrational :)

        • That’s a good point, and certainly true. I should clarify and explain that Matt (who does most of our reviews) has no view of our commercial relationships and therefore does not let that get in the way of his review. Of course he’ll be bias in some way or another, but he does his best to quantify those biases so that the reader can decide for him/herself.

        • duckingtiger

          So, just out of curiosity, what would be a good unbiased bike product review you have read, or you don’t read them at all? Most publications, are naturally sponsored by bike companies, even the venerable German Tour Magazine. Isn’t that why Ctips hires an external reviewer, and publication don’t mix sale/advertising and editorial division. (even then, I get your sentiment as many sell editorial reviews to sponsors).

          • jakub

            Well, I suppose that it is very difficult to achieve an unbiased review in a strict meaning. That would mean going through something like blinding process used in scientific studies and so on ;). Relaxing things a bit, a good unbiased review is anything where reviewer and also publisher of review has no relationship to brands being reviewed, and there is certainly no selection bias like Wade mentioned – we do not review crappy products. Of course I do read product reviews sometimes, but over years I kind of know what to expect from certain brands/product categories. I suppose that most of experienced bike riders have this kind of BS detector as well :). So product reviews serve for me mainly as what’s new on the market. If I really need to find out about some product, then I would skim through as many reviews as possible, combine this with word of mouth, see it myself and try, then decide :). But as I said, 90% cases I know what I am buying and what to expect (or at least I think so! :) – I know that the next Rapha jacket I order would be superb and size S would fit me perfectly, so there is no need to go over reviews ;)

            • duckingtiger

              I see, thanks for the laying out the thought process.

  • Chris

    “It’s okay to be irrational.”
    Thank you, thank you! I’m showing my wife this, and getting it made up on a t shirt.

    Seriously, it was most clear to that this was the case when I bought a top-tier bike earlier this year. Why did it have to be Italian with some cachet? Why not a perfectly good bike from whatever big brand probably for less? Because that’s what I wanted. Just because. And I love it.

    • Kim

      carefull, it may come right back at you with handbags and shoes

      • Chris

        Eh That’s great when it happens, actually – the hard part is getting her to indulge herself a bit more so my cycling addiction doesn’t look so out of place.

        • Dubonab1ke

          this bothers me immensely too!!

  • Ross

    Reviews of cycling products can be very subjective too, as I have found out. I have bought products (track pump is one example that comes to mind) that have been raved about by various people (mainly on online forums) and then after I get it home and start using it my reaction is “meh” Does the job but not really worth the hype.

    It works the other way too. Stuff that I find to be great other people go “meh” about. Everyone has different wants, needs, expectations and budgets and so they will like different things for different reasons.

    • jules

      it’s a track pump, they all do the same thing :-)

      • Ross

        So do carbon bike frames/wheels/handlebars/other component. So not really mate. Some have different features that may be important to some and not others (ed dual head for Presta and Schraeder) and some just work better than others. The $70 you beaut one I bought is now relegated to the back of the garage for emergency use while the $30 is much better functionality wise and is used pretty much daily. Only bad thing is the gauge is down the bottom, not near the top, making it hard for me to see the numbers. But that might not bother you. Subjective.

        • Lyre_bird

          For anybody over 50, having the gauge down the bottom makes it easier to see the numbers.

          • Matt DeMaere

            Depends on how tall you are perhaps, hehe.

            • Lyre_bird

              Over six feet in the old measure. Quite long sighted now.

          • Roger That

            Not if you’re short-sighted! (me).

          • Derek Maher

            Very true for most of we over 50 people.I am happy that I can see the 100 psi mark and work from there.

      • Chris

        Bollocks they do :-)
        Well, of course they do, but some do it better.
        I have my good bike stuff here in Singapore and “fine” stuff in Melbourne for when I’m there. The difference in usability between my two track pumps is just amazing. If I were buying again I’d have spent the extra thirty bucks and got the same as at home rather than feel like I’m risking tearing out a valve stem every time I inflate.

        • jules

          my experience is that you don’t have to spend big $$$ to get a decent track pump. my one (bought a few years ago, can’t remember brand) cost only $30, $40 tops and works like a dream.

          • Chris

            Fair enough. I’ve only owned three I can think of, and the substantially cheaper one is difficult to use compared to the other two. Should have read some reviews before buying it, I suppose, or just got another of the two I knew worked very well.

            • jules

              on the other hand, mini pumps I’ve owned – quality has varied widely.

              • Chris

                Hah! Now that’s a varied market.

                • Winky

                  Yes, for sure. Some are essentially unusable, while others are brilliant.

        • Winky

          Agree. There are some things I like in a track pump:

          1) The “feel” of the pump. I can’t decribe it, but there is way that some pumps feel when you push down (especially at high pressure) that is agreeable. This probably means that the barrel can’t be too big in diameter. Others seem vague, or like they’re fighting you.

          2) A decent chuck.
          a) Doesn’t leak, doesn’t blow off. If it uses a lever, the lever should be positive in feel, so you know it’s locked.
          b) The chuck should be able to be removed without excessive force.
          c) The chuck should be able to seal to relatively short valve stems without issues.

          3) Gauge needs to be big enough, stable, accurate and easy to read.

          4) Stable base so it doesn’t want to tip, even if you are wearing cleats. You should be able to stabilise the pump with one foot, so you can maintain a balanced stance. Some rubber on it is a nice touch, especially for cleats.

          5) Big enough handle for comfort. Non-flexy (see point 1 about “feel”).

          6) No fancy double chamber/two-stroke nonsense.

          7) A positive end stop on the downstroke, so you know you’ve finished. Not metal on metal – just a nice solid feel.

          8) The right length for comfortable pumping.

          9) Quality parts that can be serviced and replaced.

          Do I over-think this?

          • Gabriel Constantin

            Number 7, damn, I laughted at my own jokes…

          • Dave

            “Do I over-think this?”


            For a start, I could have used my $24.99 (on special) pump to fill up a few tyres in the time it took me to read your list of criteria…

      • Holby City

        They are not all equal. Specialized make strong ones that are fast and easy to pump.

        • Daniel

          Yep. I’ve only had two pumps in the last five years of proper road riding. Both were Specialized. The only complaint I had about the first one was that it broke when I accidentally drove over it in my car ;) I have never felt the slightest inclination to look at anything else.

        • Winky

          My Specialized track pump has been flawless for many years. If/when it dies I hope my next one can be as good.

  • Kayrehn

    +1 for Point 1 – I have a hunch that the first impression (so long as it’s not a 5 min round the block ride) on a proper ride can probably tell you all that you need. Spending weeks on a bike can make you lose the feel of significant differences of the test bike relative to the other bikes.

    • Chris

      Sort of – I know I can tell a massive difference between my bikes when I first go from one to another. I live OS and travel ‘home’ a few times a year. I’ll spend seven weeks at a time on one (bike #2), then go back to my usual ride and spend the first ride marvelling at the feel of bike #1. After that it just becomes ‘usual’ again.
      But long term testing is all about ‘can you live with it?’ and unearthing not so obvious issues and whatnot, which is what you want when you’re reading reviews and are about to stump up thousands for the product.

      • I always reach a point with every bike that I’ve reviewed where I feel as if I “understand” it. Some bikes reveal their personality quickly while others need more time. Then I go back to my own bike for a couple of days before trying out the review bike again, just to confirm all of my impressions. From there, I feel pretty confident extrapolating forwards to decide if I could live with it. Not as good as the real thing, but I think of every bike (and any other product) as something I have bought so I can decide on its value.

        • Chris

          Thanks for the insight into your method – I really only meant “long term” in that it’s more than a decent long ride and you get to see a little about conducting maintenance or whatever (chainstay-mounted brakes ftw).

  • Ralph

    Interesting first point on initial impressions. Reminds me of Richard Parry-Jones’s 50 metre test..

  • Justme

    Thank you Wade and Matt, especially for Rule #8 “There is no industry conspiracy to rip-off consumers”!! There is a small minority of extremely annoying grizzlers / cynics who infiltrate each and every review with a conspiracy theory. You guys tirelessly respond, but it’s frustrating to have to read them on every review comment section (and yes, generally, the comments are good and help make an informed consumer choice so I can’t just ignore them).

  • Spider

    Matt, I understand that you probably don’t get to spend to long on your own personal bike….but what set up do you run?

    or another question….here’s a blank chequebook….go nuts…..what do you buy (that you’ve ridden and appreciated)…

    thank you

    • I’ve been riding a Ridley Damocles with a 10 speed Chorus groupset for the last 7-8 years and it has served as a pretty good baseline for comparison with the bikes I’m reviewing. However, I’m in the final stages of putting together a new bike for myself, and while didn’t have a blank chequebook, it was the first time where budget was a secondary consideration. I don’t want to spoil the surprise until I have pictures of the finished bike, but I will say that it’s quite traditional compared to many of the bikes I’ve reviewed.

      • Spider

        Thanks Matt. Be interested to see what this new project is! I love my record 10 speed from the same era – very…..campy!

        So….with comments like that ‘quite traditional’….it isn’t an Pinarello F8 or canyon Aeroad…

        (just kidding, I dislike those guys who try and guess the secret pro…I like a bit of suspense)

  • purpletezza

    I really liked the “ok to be irrational” comment. My prime criteria for selecting a bike frame/bike to buy is the frame’s paint design and colourway, ie how good does it look? Frame weight, aero, blah blah blah doesn’t really sway me.

    • Winky

      “colourway”? Do you mean “colour scheme” or perhaps just “colour”? And you kids get off my lawn.

      • ZigaK

        BSNYC? :)

        • Winky

          I do lurk there, yes. I’ve been a big fan for years.

  • Anon N + 1

    “riders won’t sense the energy savings though; they’ll just feel like they can go a little faster.”
    This observation is a corollary of 10.

    • Anon N + 1

      “of Rule 10” Sorry for the typo

    • Dave

      “This observation is a corollary of 10.”

      And a correlation of 1.0 – but the jury is out on the causation ?

  • Chiwode

    Super Record and Chorus perform the same. I own both. But Super Record lasts forever. I won’t even say how many miles I got out of my last SR cassette because no one would believe me. So there’s that. Hard to review longevity.

    • Frank

      I get 11 ~ 12.000 km out of Record chains – and change them before it’s really necessary!

  • ZigaK

    There’s no room for half baked products – press fit bottom bracket?

    • Winky

      Totally agree. This cost-saving crappy engineering “solution” deserves much more scorn than it gets. Same with carbon clinchers – heavy, don’t brake well, safety concerns, but hey, at least they’re expensive.

  • Superpilot

    For your sake I’m glad you didn’t mention it, but I will say that you neglected to mention rule 10 – the internet is full of hypersensitive, over thinking, negatively biased, constantly suspicious characters who like to brew storms in tea cups, and thunder and lightning with their keystrokes. Thank you for the work you do Matt and all at CT, it’s a thankless job when the squeaky wheels make the most noise so easily :) From what I’ve seen you must be constant *rolleyes* *facepalm* *head+brickwall*, it is appreciated.

  • VK

    As a consumer of Cyclingtips, my consumer confidence has dropped after reading the recent review articles but I have to admit that it will be back on an upward trend after reading this article. Thanks Wade and Cyclingtips for always responding to address our dissatisfaction and complaints.

    • Thanks VK! Earning yours and everyone else’s trust is something we’ll never be able to rest on.

  • Derek Maher

    Good article Matt.Ist lesson.You won’t please everyone no matter what you write.
    Keep up the good work.

  • Andy B

    Out of curiosity when you test bikes do you try to imitate your ideal riding position the same for each bike?
    And if so do you find this difficult to achieve?

    As someone who owns many multiple bikes I find it difficult to get them all set up spot on for me
    I guess the hardest thing being stem length/handlebar width/saddle and saddle height all differ slightly and without having a stockpile of gear it can be difficult to achieve the right fit between brands and frame geometries
    Just curious if you try and adapt each bike to your fit or you just test them based on being near enough


Pin It on Pinterest

October 26, 2016
October 25, 2016
October 24, 2016
October 22, 2016