Hiding pages from search

I'm working with a department that would like to link to internal content on their website in their email communications, without cluttering the experience for external audiences. The two ways we could see doing this is hiding those pages from search engines and/or creating password-protected content on their website. Is any of that currently possible?
4 people like
this idea
+1
Reply
  • The answer depends on how secret you want to keep the content and to whom you want to make it available. OpenScholar does not have a solution for all use cases but has options for most.

    You can add a sub site with a different Site Visibility setting than the main, public site.

    If you publish a page to a public site but don't link to it from a public page (it doesn't appear in any menus, it's not added to a section, no taxonomy added, etc.), Google and other search engines out in the world can't find their way to the page to list it. The OpenScholar site's own search box *can* find and list such pages if someone searches for a word that appears on the page (e.g. if you have "2015 Award Winners" page you don't want to make public, someone can find it if they use your site's search box and search for "2015," "award," "winners," etc.).

    You can edit a Post, go to the Publishing options and uncheck "Published to this site." That will make it not show up in the OpenScholar site's search and the page won't be reachable unless you login using the Admin Login (meaning you have an OpenScholar account that's listed on the site's /cp/users page).
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Thanks, Curtis.

    "If you publish a page to a public site but don't link to it from a public page (it doesn't appear in any menus, it's not added to a section, no taxonomy added, etc.), Google and other search engines out in the world can't find their way to the page to list it."

    My understanding is that Google will find everything unless you tell it not to index the page. (https://support.google.com/webmasters...) Is there a way to add a 'noindex' tag to a page in OpenScholar?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Search engines can't find *everything.* Say I create a new page with the URL example.harvard.edu/0cfa9f600839f57e90e5559b8ee54864. I don't add it to my site's menu, I don't add it to a section, I don't link to it from the body of another page or anywhere else. How would Google find it?

    If your OpenScholar site is configured to use Google Analytics, I suppose it's possible that GA could share what it knows with Google Search by hunch is that doesn't happen.

    The noindex meta tag described at that link goes into the portion of the web page which is not a part of the page we can directly edit. The OpenScholar team could add a feature, perhaps another checkbox under Publishing Options, that would insert a noindex meta tag for you but I'd much rather see features added that provide real access control to specific content.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • dorian (Official Rep) March 05, 2015 16:44
    So far the discussion has been about 'searchability' of the pages... Catherine, if the pages were password-protected, what were you hoping the login authentication be? Individual accounts that the site owner creates, or anyone at Harvard, or just viewable by anyone with a link?

    Is this a case of wanting both login AND be non searchable, or is either case acceptable?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • In the use-case I'm thinking of, the information being shared via email would not be confidential. So, I think that a great option would be if the page is viewable by anyone with the link. It might even be a deterrent if people need to log in to see the page.

    Ideally, we'd want both a way to limit access and be non-searchable. The pages should be easy for Harvard people to access from the context of an internal email, but not clutter the website (or search results) for the primary audience of the website.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Catherine,

    I think adding the ability for a page to be un-indexable is a good idea. We don't currently however have anything like that. Your only real option is to create a separate site with the privacy set to "Anyone with a link". In this case the whole site is opted out from indexing.

    -Richard
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated