{"componentChunkName":"component---src-components-public-workflows-single-workflow-index-tsx","path":"/ready-made-workflow-library/crawl-your-website-&-create-an-xml-sitemap","result":{"pageContext":{"result":{"workflow":{"id":50,"name":"Crawl your website & create an XML sitemap","status":null,"started_at":"1634285288035","frequency":"","created_at":"1634285404721","updated_at":"1634285404721","description":"XML Sitemaps help search engines discover your content and index it. This workflow crawls an entire website just like Google and generates an XML sitemap file, no coding required.","meta_title":"Create an XML sitemap by crawling your website - Hexomatic workflow","meta_description":"<p>XML Sitemaps help search engines discover your content and index it. The file itself provides a structured list of all pages found on a website facilitating the discovery of new or updated pages.</p>\n\n<p>With Hexomatic you can generate XML sitemap files in minutes, no coding or software required.</p>\n\n<p>This workflow enables you to specify any domain and perform an in-depth crawl to discover all the publicly accessible pages and create a standard XML file you can download.</p>","image":"https://storage.googleapis.com/hexomatic-automation-icons/public-workflows-illustrations/Crawl%20your%20website%20%26%20create%20an%20XML%20sitemap.jpg","categories":[{"id":2,"name":"Marketing"}],"steps":[{"id":297,"setting":"{\"automation_id\":81,\"dynamic_settings\":{\"keys\":[{\"key\":null,\"settings\":{\"limit\":100,\"proxy\":{\"type\":\"\",\"country_code\":\"\"},\"browser\":false,\"url_type\":\"internal\",\"page_type\":\"page\",\"ignore_urls\":\"\"},\"inputValue\":\"hexomatic.com\"}]},\"output_keys_data\":[\"_source_url\",\"_crawled_url\",\"_crawler_detected_on\",\"_crawler_status\",\"_crawler_link_type\",\"_crawler_url_type\"],\"predefined_settings\":{\"limit\":100,\"url_type\":\"internal\",\"page_type\":\"page\"},\"integration_settings\":null,\"output_keys_data_with_types\":\"{\\\"_source_url\\\":\\\"url\\\",\\\"_crawled_url\\\":\\\"url\\\",\\\"_crawler_detected_on\\\":\\\"url\\\",\\\"_crawler_status\\\":\\\"number\\\",\\\"_crawler_link_type\\\":\\\"text\\\",\\\"_crawler_url_type\\\":\\\"text\\\"}\"}","index":0,"created_at":"1631606440232","updated_at":null,"automation":{"id":81,"name":"Crawler","url":"Crawler","meta_title":"Website Crawler | Links Extractor Automation","meta_description":"Use our website crawler to extract pages, source URLs, and internal and external links. It shows the response code for each URL and consumes an automation credit per URL.","short_description":"Crawl any website extracting pages, source URLs, and links (internal and external)","long_description":"Crawl any website extracting pages, source URLs, and links (internal and external).\nThis automation shows the response code for each URL and consumes an automation credit per URL.\nCan be used as a starting point for other automations including the SEO meta tag, email scraper, screenshot, image or file downloader, keyword finder or audit automations. (1 automation credit per crawled URL)\n\n\n\n\n\n","long_description_details":"Our crawler automation will help you to crawl any website extracting pages, source URLs, and links (both internal and external).\n<br />This automation shows the response code for each URL and consumes an automation credit per URL.\n<br />Can be used as a starting point for other automations including the SEO meta tag, email scraper, screenshot, image or file downloader, keyword finder or audit automations.\n<br />**Different ways to use our Crawler automation:**\n<br />\n> * This automation can be used as a starting point of your Hexomatic workflow.\n<br />\n\n**How to use our Crawler automation:**\n<br />Step 1: Create a blank workflow.\nStep 2: Add the Crawler automation. After enter the source URL, the URLs to be ignored, URL types, the proxy mode and set the URL limit.\nStep 3: Click continue to run or schedule the workflow.","has_source":null,"icon_url":"https://storage.googleapis.com/hexomatic-automation-icons/crawler-automation.svg"}},{"id":298,"setting":"{\"automation_id\":23,\"dynamic_settings\":{\"keys\":[{\"key\":\"_crawled_url\",\"settings\":null,\"inputValue\":null}]},\"output_keys_data\":[\"_source_url\",\"_crawled_url\",\"_crawler_detected_on\",\"_crawler_status\",\"_crawler_link_type\",\"_crawler_url_type\",\"_generated_sitemap_url\"],\"predefined_settings\":null,\"integration_settings\":null,\"output_keys_data_with_types\":\"{\\\"_source_url\\\":\\\"url\\\",\\\"_crawled_url\\\":\\\"url\\\",\\\"_crawler_detected_on\\\":\\\"url\\\",\\\"_crawler_status\\\":\\\"number\\\",\\\"_crawler_link_type\\\":\\\"text\\\",\\\"_crawler_url_type\\\":\\\"text\\\",\\\"_generated_sitemap_url\\\":\\\"url\\\"}\"}","index":1,"created_at":"1631606440232","updated_at":null,"automation":{"id":23,"name":"XML sitemap generator","url":"xml-sitemap-generator","meta_title":"Generate xml sitemaps online | Sitemap Generator","meta_description":"Need to create an XML sitemap? This automation enables you to generate an XML sitemap from your structured data in just a few clicks. ","short_description":"Generate one XML sitemap from all inputted URLs","long_description":"Create an XML sitemap from your structured data  (* Each request is rounded up to the nearest increment of 100 URLs).","long_description_details":"XML sitemap serves as a roadmap of your website, leading Google to all your essential pages. For example, you want Google to crawl every important page of your website. However, often, pages end up without any internal links that point to them. This makes the pages hard to find. Having a sitemap will significantly speed up the content discovery process. \n<br />Our Hexomatic sitemap generator will generate an XML sitemap from your structured data (* Each request is rounded up to the nearest increment of 100 URLs).\n<br />**Different ways to use our XML sitemap generator:**\n<br />\n> * Via Data input automation (using the Manual paste / List of inputs) to generate XML sitemap from the list of URLs.\n> * Via Data input automation (using the Upload file / CSV file option) to generate XML sitemap from URLs in a local CSV file.\n<br />\n\n**How to use our XML sitemap generator via Data input:**\n<br />Step 1: Create a blank workflow and select Data input as the starting point.\nStep 2: Add the XML sitemap generator automation, selecting Data input as the source. \nStep 3: Click continue to run or schedule the workflow.\n<br />You can export the results in CSV or Google Sheets.","has_source":null,"icon_url":"https://storage.googleapis.com/hexomatic-automation-icons/xml-sitemap-generator.svg"}}],"tags":["Marketing"]},"prevPath":"/ready-made-workflow-library/research-backlink-partners-&-contact-details-via-list","nextPath":"/ready-made-workflow-library/discover-email-addresses-for-a-list-of-websites"}}},"staticQueryHashes":["3915125517"]}