
Say Goodbye to Null Columns: The Power Query Hack You Need
Excel Off The Grid takes you through a dynamic solution for removing null columns that adapts to the ever-changing nature of your datasets. By using tools like the `Table.Profile` function, you can identify and filter out irrelevant columns in a way that's both automated and scalable. Whether you're working with a single workbook or managing multiple datasets, this method ensures your data cleaning process is not only faster but also smarter. And if you're looking to take it a step further, we'll explore how to create a reusable custom function that simplifies repetitive tasks. Imagine the possibilities when your data workflows are streamlined, consistent, and free of clutter. Dynamic Null Column Removal Why Opt for a Dynamic Approach?
When working with datasets that evolve over time, hardcoding column names can create unnecessary limitations and increase maintenance efforts. Power Query's dynamic tools, such as the `Table.Profile` function, provide a flexible and automated solution for analyzing and cleaning data. By dynamically identifying and removing null columns, you ensure that only meaningful data is retained, regardless of how your dataset changes. This approach not only saves time but also enhances the scalability of your data workflows. Steps to Dynamically Remove Null Columns
Removing null columns in Power Query is a straightforward and systematic process. Follow these steps to clean your dataset dynamically: Load your dataset: Begin by importing your data into Power Query to initiate the cleaning process.
Begin by importing your data into Power Query to initiate the cleaning process. Remove unnecessary steps: Delete hardcoded steps, such as 'Change Type,' to maintain flexibility and avoid static dependencies in your query.
Delete hardcoded steps, such as 'Change Type,' to maintain flexibility and avoid static dependencies in your query. Generate a table summary: Use the `Table.Profile` function to create a summary of your dataset. This summary provides key metrics, including column counts and null counts for each column.
Use the `Table.Profile` function to create a summary of your dataset. This summary provides key metrics, including column counts and null counts for each column. Identify columns with meaningful data: Add a custom column to the summary table to compare the `Count` and `Null Count` values. This step helps pinpoint columns that contain actual data.
Add a custom column to the summary table to compare the `Count` and `Null Count` values. This step helps pinpoint columns that contain actual data. Filter out null columns: Apply a filter to exclude columns where the `Count` equals the `Null Count`, effectively isolating columns with meaningful data.
Apply a filter to exclude columns where the `Count` equals the `Null Count`, effectively isolating columns with meaningful data. Apply the filtered results: Use the filtered summary to dynamically remove null columns from your original dataset, making sure only relevant data remains.
This method ensures your data cleaning process is efficient, adaptable, and scalable, even for complex datasets with varying structures. Power Query : Instantly Remove All Null Columns
Watch this video on YouTube.
Here are more detailed guides and articles that you may find helpful on Power Query. Creating a Custom Function for Reusability
To further streamline your workflow, you can convert this process into a reusable custom function. This allows you to apply the null column removal process across multiple datasets without recreating the query each time. Here's how to create a custom function: Duplicate your query: Start by copying the query you created for removing null columns.
Start by copying the query you created for removing null columns. Access the Advanced Editor: Open the Advanced Editor in Power Query to modify the query code.
Open the Advanced Editor in Power Query to modify the query code. Define a function: Replace static references in the query with a dynamic table input parameter. This step generalizes the query, making it adaptable to different datasets.
Replace static references in the query with a dynamic table input parameter. This step generalizes the query, making it adaptable to different datasets. Save the function: Assign a name to the function and save it. This makes the function accessible for use across various datasets or workbooks.
By creating a custom function, you can significantly reduce the time and effort required for repetitive data cleaning tasks, while making sure consistency and accuracy. Advantages of a Reusable and Dynamic Process
Implementing a reusable custom function for removing null columns offers several key benefits: Time efficiency: Save time by applying the function to multiple datasets without needing to recreate the query for each one.
Save time by applying the function to multiple datasets without needing to recreate the query for each one. Reduced errors: Minimize the risk of mistakes that can occur during manual data cleaning processes.
Minimize the risk of mistakes that can occur during manual data cleaning processes. Scalability: Adapt the function to datasets of varying sizes and structures, making sure it remains effective as your data grows or changes.
Adapt the function to datasets of varying sizes and structures, making sure it remains effective as your data grows or changes. Consistency: Maintain a standardized approach to data cleaning, which is particularly useful when working with large teams or multiple projects.
By using Power Query's dynamic capabilities and creating reusable functions, you can ensure your data cleaning workflows remain robust, efficient, and adaptable to evolving requirements. Streamlining Data Cleaning with Power Query
Dynamically removing null columns in Power Query is a practical and effective way to streamline your data cleaning process. By using the `Table.Profile` function and creating a reusable custom function, you can handle datasets of any size or complexity with ease. This approach not only saves time but also ensures your data remains clean, meaningful, and ready for analysis. Whether you're managing a single dataset or working across multiple workbooks, this method provides a reliable solution for maintaining high-quality data.
Media Credit: Excel Off The Grid Filed Under: Guides, Top News
Latest Geeky Gadgets Deals
Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.
Hashtags

Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles


The Guardian
an hour ago
- The Guardian
AI companies start winning the copyright fight
Hello, and welcome to TechScape. If you need me after this newsletter publishes, I will be busy poring over photos from Jeff Bezos and Lauren Sanchez's wedding, the gaudiest and most star-studded affair to disrupt technology news this year. I found it a tacky and spectacular affair. Everyone who was anyone was there, except for Charlize Theron, who, unprompted, said on Monday: 'I think we might be the only people who did not get an invite to the Bezos wedding. But that's OK, because they suck and we're cool.' Last week, tech companies notched several victories in the fight over their use of copyrighted text to create artificial intelligence products. Anthropic: A US judge has ruled that Anthropic, maker of the Claude chatbot, use of books to train its artificial intelligence system – without permission of the authors – did not breach copyright law. Judge William Alsup compared the Anthropic model's use of books to a 'reader aspiring to be a writer.' And the next day, Meta: The US district judge Vince Chhabria, in San Francisco, said in his decision on the Meta case that the authors had not presented enough evidence that the technology company's AI would cause 'market dilution' by flooding the market with work similar to theirs. The same day that Meta received its favorable ruling, a group of writers sued Microsoft, alleging copyright infringement in the creation of that company's Megatron text generator. Judging by the rulings in favor of Meta and Anthropic, the authors are facing an uphill battle. These three cases are skirmishes in the wider legal war over copyrighted media, which rages on. Three weeks ago, Disney and NBC Universal sued Midjourney, alleging that the company's namesake AI image generator and forthcoming video generator made illegal use of the studios' iconic characters like Darth Vader and the Simpson family. The world's biggest record labels – Sony, Universal, and Warner – have sued two companies that make AI-powered music generators, Suno and Udio. On the textual front, the New York Times' suit against OpenAI and Microsoft is ongoing. The lawsuits over AI-generated text were filed first, and, as their rulings emerge, the next question in the copyright fight is whether decisions about one type of media will apply to the next. 'The specific media involved in the lawsuit – written works versus images versus videos versus audio – will certainly change the fair use analysis in each case,' said John Strand, a trademark and copyright attorney with the law firm Wolf Greenfield. 'The impact on the market for the copyrighted works is becoming a key factor in the fair use analysis, and the market for books is different than that for movies.' To Strand, the cases over images seem more favorable to copyright holders, as the AI models are allegedly producing identical images to the copyrighted ones in the training data. A bizarre and damning fact was revealed in the Anthropic ruling, too: the company had pirated and stored some 7m books to create a training database for its AI. To remediate its wrongdoing, the company bought physical copies and scanned them, digitizing the text. Now the owner of 7 million physical books that no longer held any utility, Anthropic destroyed them. The company bought the books, diced them up, scanned the text, and threw them away, Ars Technica reports. There are less destructive ways to digitize books, but they are slower. The AI industry is here to move fast and break things. Anthropic laying waste to millions of books presents a crude literalization of the ravenous consumption of content necessary for AI companies to create their products. Google's emissions up 51% as AI electricity demand derails efforts to go green Inside a plan to use AI to amplify doubts about the dangers of pollutants Two stories I wrote about last week saw significant updates in the ensuing days. The website for Trump's gold phone, 'T1', has dropped its 'Made in America' pledge in favor of 'proudly American' and 'brought to life in America', per the Verge. Trump seems to have followed the example of Apple, which skirts the issue of origin but still emphasizes the American-ness of iPhones by engraving them with 'Designed in California.' What is unsaid: Assembled in China or India, and sourced from many other countries. It seems Trump and his family have opted for a similar evasive tagline, though it's been thrown into much starker relief by their original promise. The third descriptor that now appears on Trump's phone site, 'American-Proud Design', seems most obviously cued by Apple. The tagline 'Made in the USA' carries legal weight. Companies have faced lawsuits over just how many of their products' parts were produced in the US, and the US' main trade regulator has established standards by which to judge the actions behind the slogan. It would be extremely difficult for a smartphone's manufacturing history to measure up to those benchmarks, by the vast majority of expert estimations. Though Trump intends to repatriate manufacturing in the US with his sweeping tariffs, he seems to be learning just what other phone companies already know. It is complicated and limiting to make a phone solely in the US, and doing so forces severe constraints on the final product. Read last week's newsletter about the gold Trump phone. Last week, I wrote about Pornhub's smutty return to France after a law requiring online age verification was suspended there. This week, the US supreme court ruled in favor of an age-check law passed in Texas. Pornhub has blocked access to anyone in Texas in protest for the better part of two years, as it did in France for three weeks. Clarence Thomas summed up the court's reasoning: 'HB 1181 simply requires adults to verify their age before they can access speech that is obscene to children,' Clarence Thomas wrote in the court's 6-3 majority opinion. 'The statute advances the state's important interest in shielding children from sexually explicit content. And, it is appropriately tailored because it permits users to verify their ages through the established methods of providing government-issued identification and sharing transactional data.' Elena Kagan dissented alongside the court's two other liberal justices. The ruling affirms not only Texas's law but the statutes of nearly two dozen states that have implemented online age checks. The tide worldwide seems to be shifting away from allowing freer access to pornography as part of a person's right to free expression and more towards curtailing Experts believe the malleable definition of obscenity – the Texas law requires an age check for any site whose content is more than a third sexual material – will be weaponized against online information on sexual health, abortion or LGBTQ identity, all in the name of child protection. 'It's an unfortunate day for the supporters of an open internet,' said GS Hans, professor at Cornell Law School. 'The court has made a radical shift in free speech jurisprudence in this case, though it doesn't characterize its decision that way. By upholding the limits on minors' access to obscenity – a notoriously difficult category to define – that also creates limits on adult access, we can expect to see states take a heavier hand in regulating content.' I'll be closely watching what happens in July when Pornhub willingly implements age checks in compliance with the Online Services Act. Read more: UK study shows 8% of children aged eight to 14 have viewed online pornography Number of new UK entry-level jobs has dived since ChatGPT launch – research Fake, AI-generated videos about the Diddy trial are raking in millions of views on YouTube Denmark to tackle deepfakes by giving people copyright to their own features New features are a dime a dozen, but even a small tweak to the most popular messaging app in the world may amount to a major shift. WhatsApp will begin showing you AI-generated summaries of your unread messages, per the Verge. Apple tried message summaries. They did not work. The company pulled them. For a firm famed for its calculated and controlled releases, the retraction of the summaries was a humiliation. The difference between Apple and Meta, though, is that Meta has consistently released AI products for multiple years now. In other AI news, I am rarely captivated by new technologies, but a recent release by Google's DeepMind AI laboratory seems promising for healthcare. Google DeepMind has released AlphaGenome, an AI meant to 'comprehensively and accurately predicts how single variants or mutations in human DNA sequences impact a wide range of biological processes regulating genes,' per a press release. The creators of AlphaGenome previously won the Nobel prize in chemistry for AlphaFold, a software that predicts the structures of proteins. A major question that hovers over Crispr, another Nobel-winning innovation, is what changes in a person when a genetic sequence is modified. AlphaGenome seems poised to assist in solving that mystery. Disabled Amazon workers in corporate jobs allege 'systemic discrimination' Six arrested at protest of Palantir, tech company building deportation software for Trump admin Online hacks to offline heists: crypto leaders on edge amid increasing attacks 'Lidar is lame': why Elon Musk's vision for a self-driving Tesla taxi faltered 'It's like being walled in': young Iranians try to break through internet blackout


Reuters
an hour ago
- Reuters
Exclusive: Scale AI's bigger rival Surge AI seeks up to $1 billion capital raise, sources say
July 1 (Reuters) - Surge AI, a data-labeling firm that competes with Scale AI, has hired advisors to raise as much as $1 billion in the first capital raising in the firm's history, sources told Reuters, as it seeks to capitalize on growing user demand amid Scale AI's recent customer exodus. The company, founded by former Google and Meta engineer Edwin Chen, is targeting a valuation of over $15 billion, sources said, cautioning that the talks are still in early stages and the final number could be higher. The funding would be a mix of primary and secondary capital that provides liquidity for the employees. Surge AI, which has been profitable and bootstrapped by Chen, has raked in over $1 billion in revenue last year, bigger than its better-known competitor Scale AI, which reported $870 million in revenue over the same period of time. In comparison, Scale AI was valued at $14 billion in a funding round last year, and was mostly recently valued at nearly $29 billion when Meta invested for a 49% stake in the company and poached its CEO Alexandr Wang to be its chief AI officer to lead its new Superintelligence Labs. Surge AI declined to comment. Like other Scale AI competitors, Surge AI is benefiting from Scale AI's customer losses following Meta's investment. This includes OpenAI and Scale's largest customer, Google, who are now planning to move away from the platform over concerns that doing business with Scale could expose their research priorities to Meta. Scale has said its business remains strong, and it is committed to protecting customer data. Surge AI's quiet yet meteoric rise has positioned it as one of the largest players in the crowded data labeling industry, defying the typical Silicon Valley playbook of raising massive rounds of venture capital to fuel growth. Founded in 2020, the San Francisco-based company has largely operated under the radar, known for its premium, high-end data labeling services used by top AI labs, including Google, OpenAI and Anthropic. As reinforcement learning from human feedback (RLHF) has become more important in training advanced AI systems, the demand for meticulously labeled, nuanced datasets has grown. Surge AI has capitalized on this trend by appealing to a network of highly skilled contractors instead of large pools of low-wage labor. The outsized funding of Surge would be a test of investor interest in the data labeling sector. Some investors view data labeling as an ongoing necessity for AI development, predicting a continued demand from leading AI labs. Others express concern that the industry's low margins and reliance on human labor could make it vulnerable to automation, as AI technology advances and the need for manual annotation diminishes.


The Guardian
2 hours ago
- The Guardian
AI companies start winning the copyright fight
Hello, and welcome to TechScape. If you need me after this newsletter publishes, I will be busy poring over photos from Jeff Bezos and Lauren Sanchez's wedding, the gaudiest and most star-studded affair to disrupt technology news this year. I found it a tacky and spectacular affair. Everyone who was anyone was there, except for Charlize Theron, who, unprompted, said on Monday: 'I think we might be the only people who did not get an invite to the Bezos wedding. But that's OK, because they suck and we're cool.' Last week, tech companies notched several victories in the fight over their use of copyrighted text to create artificial intelligence products. Anthropic: A US judge has ruled that Anthropic, maker of the Claude chatbot, use of books to train its artificial intelligence system – without permission of the authors – did not breach copyright law. Judge William Alsup compared the Anthropic model's use of books to a 'reader aspiring to be a writer.' And the next day, Meta: The US district judge Vince Chhabria, in San Francisco, said in his decision on the Meta case that the authors had not presented enough evidence that the technology company's AI would cause 'market dilution' by flooding the market with work similar to theirs. The same day that Meta received its favorable ruling, a group of writers sued Microsoft, alleging copyright infringement in the creation of that company's Megatron text generator. Judging by the rulings in favor of Meta and Anthropic, the authors are facing an uphill battle. These three cases are skirmishes in the wider legal war over copyrighted media, which rages on. Three weeks ago, Disney and NBC Universal sued Midjourney, alleging that the company's namesake AI image generator and forthcoming video generator made illegal use of the studios' iconic characters like Darth Vader and the Simpson family. The world's biggest record labels – Sony, Universal, and Warner – have sued two companies that make AI-powered music generators, Suno and Udio. On the textual front, the New York Times' suit against OpenAI and Microsoft is ongoing. The lawsuits over AI-generated text were filed first, and, as their rulings emerge, the next question in the copyright fight is whether decisions about one type of media will apply to the next. 'The specific media involved in the lawsuit – written works versus images versus videos versus audio – will certainly change the fair use analysis in each case,' said John Strand, a trademark and copyright attorney with the law firm Wolf Greenfield. 'The impact on the market for the copyrighted works is becoming a key factor in the fair use analysis, and the market for books is different than that for movies.' To Strand, the cases over images seem more favorable to copyright holders, as the AI models are allegedly producing identical images to the copyrighted ones in the training data. A bizarre and damning fact was revealed in the Anthropic ruling, too: the company had pirated and stored some 7m books to create a training database for its AI. To remediate its wrongdoing, the company bought physical copies and scanned them, digitizing the text. Now the owner of 7 million physical books that no longer held any utility, Anthropic destroyed them. The company bought the books, diced them up, scanned the text, and threw them away, Ars Technica reports. There are less destructive ways to digitize books, but they are slower. The AI industry is here to move fast and break things. Anthropic laying waste to millions of books presents a crude literalization of the ravenous consumption of content necessary for AI companies to create their products. Google's emissions up 51% as AI electricity demand derails efforts to go green Inside a plan to use AI to amplify doubts about the dangers of pollutants Two stories I wrote about last week saw significant updates in the ensuing days. The website for Trump's gold phone, 'T1', has dropped its 'Made in America' pledge in favor of 'proudly American' and 'brought to life in America', per the Verge. Trump seems to have followed the example of Apple, which skirts the issue of origin but still emphasizes the American-ness of iPhones by engraving them with 'Designed in California.' What is unsaid: Assembled in China or India, and sourced from many other countries. It seems Trump and his family have opted for a similar evasive tagline, though it's been thrown into much starker relief by their original promise. The third descriptor that now appears on Trump's phone site, 'American-Proud Design', seems most obviously cued by Apple. The tagline 'Made in the USA' carries legal weight. Companies have faced lawsuits over just how many of their products' parts were produced in the US, and the US' main trade regulator has established standards by which to judge the actions behind the slogan. It would be extremely difficult for a smartphone's manufacturing history to measure up to those benchmarks, by the vast majority of expert estimations. Though Trump intends to repatriate manufacturing in the US with his sweeping tariffs, he seems to be learning just what other phone companies already know. It is complicated and limiting to make a phone solely in the US, and doing so forces severe constraints on the final product. Read last week's newsletter about the gold Trump phone. Last week, I wrote about Pornhub's smutty return to France after a law requiring online age verification was suspended there. This week, the US supreme court ruled in favor of an age-check law passed in Texas. Pornhub has blocked access to anyone in Texas in protest for the better part of two years, as it did in France for three weeks. Clarence Thomas summed up the court's reasoning: 'HB 1181 simply requires adults to verify their age before they can access speech that is obscene to children,' Clarence Thomas wrote in the court's 6-3 majority opinion. 'The statute advances the state's important interest in shielding children from sexually explicit content. And, it is appropriately tailored because it permits users to verify their ages through the established methods of providing government-issued identification and sharing transactional data.' Elena Kagan dissented alongside the court's two other liberal justices. The ruling affirms not only Texas's law but the statutes of nearly two dozen states that have implemented online age checks. The tide worldwide seems to be shifting away from allowing freer access to pornography as part of a person's right to free expression and more towards curtailing Experts believe the malleable definition of obscenity – the Texas law requires an age check for any site whose content is more than a third sexual material – will be weaponized against online information on sexual health, abortion or LGBTQ identity, all in the name of child protection. 'It's an unfortunate day for the supporters of an open internet,' said GS Hans, professor at Cornell Law School. 'The court has made a radical shift in free speech jurisprudence in this case, though it doesn't characterize its decision that way. By upholding the limits on minors' access to obscenity – a notoriously difficult category to define – that also creates limits on adult access, we can expect to see states take a heavier hand in regulating content.' I'll be closely watching what happens in July when Pornhub willingly implements age checks in compliance with the Online Services Act. Read more: UK study shows 8% of children aged eight to 14 have viewed online pornography Number of new UK entry-level jobs has dived since ChatGPT launch – research Fake, AI-generated videos about the Diddy trial are raking in millions of views on YouTube Denmark to tackle deepfakes by giving people copyright to their own features New features are a dime a dozen, but even a small tweak to the most popular messaging app in the world may amount to a major shift. WhatsApp will begin showing you AI-generated summaries of your unread messages, per the Verge. Apple tried message summaries. They did not work. The company pulled them. For a firm famed for its calculated and controlled releases, the retraction of the summaries was a humiliation. The difference between Apple and Meta, though, is that Meta has consistently released AI products for multiple years now. In other AI news, I am rarely captivated by new technologies, but a recent release by Google's DeepMind AI laboratory seems promising for healthcare. Google DeepMind has released AlphaGenome, an AI meant to 'comprehensively and accurately predicts how single variants or mutations in human DNA sequences impact a wide range of biological processes regulating genes,' per a press release. The creators of AlphaGenome previously won the Nobel prize in chemistry for AlphaFold, a software that predicts the structures of proteins. A major question that hovers over Crispr, another Nobel-winning innovation, is what changes in a person when a genetic sequence is modified. AlphaGenome seems poised to assist in solving that mystery. Disabled Amazon workers in corporate jobs allege 'systemic discrimination' Six arrested at protest of Palantir, tech company building deportation software for Trump admin Online hacks to offline heists: crypto leaders on edge amid increasing attacks 'Lidar is lame': why Elon Musk's vision for a self-driving Tesla taxi faltered 'It's like being walled in': young Iranians try to break through internet blackout