Home Blog Page 37

The Impact of Artificial Intelligence on Podcast Quality

0

The Impact of Artificial Intelligence on Podcast Quality

Introduction

The world of podcasting has undergone significant changes in recent years, largely due to the rise of Artificial Intelligence (AI). What started as a simple medium for storytelling and information sharing has evolved into a multi-faceted industry that offers vast content across genres. With the rapid pace of technological advancement, AI has become a key player in transforming podcast creation and listening experiences.

Today, AI podcast workflow tools are not just a luxury for podcast creators—they are essential. From content generation to post-production, AI is helping to streamline every phase of the podcasting journey. But how exactly is AI impacting the quality of podcasts? And more importantly, what does this mean for the future of podcasting? Let’s dive in and explore how AI is reshaping this creative industry.

The Role of AI in Podcast Production

The production process of creating a podcast has become more efficient thanks to AI. Whether it’s AI transforming podcast creation or streamlining editing, AI is playing a huge role in this transformation. AI tools can automatically write scripts, generate ideas, and even adapt the tone and style of content to suit specific audiences.

Many creators are already using free podcast generators to produce high-quality content quickly. These tools often use AI algorithms to create podcast scripts based on topics, trends, or even user input. AI can help ensure that podcast content is fresh, relevant, and well-organized.

Some advanced AI podcast workflow tools can also assist with voiceovers, ensuring that the tone matches the desired style. Whether you’re a beginner or a seasoned professional, the use of these tools helps reduce the time and effort needed to create polished, engaging podcasts.

Enhancing Audio Quality with AI

One of the most noticeable impacts of AI in podcasting is the improvement in audio quality. Audio editing can be time-consuming, requiring precise adjustments to ensure clarity. AI, however, can automate many of these tasks, making the process more efficient.

AI transforming podcast creation includes sophisticated tools for audio enhancement. These tools can remove background noise, balance sound levels, and even enhance the clarity of voices. AI algorithms analyze the audio track and suggest changes to improve overall sound quality, making it more professional and polished.

Using AI for audio editing has significantly cut down on production time, allowing creators to focus more on content rather than technical details. Podcast editors now benefit from features such as automatic volume adjustment, noise reduction, and voice enhancement, all driven by AI. The result? A higher-quality listening experience for the audience.

Personalized Podcast Experience

AI doesn’t just improve the technical aspects of podcasting; it also plays a significant role in personalizing the experience for listeners. AI in podcast content creation can help tailor recommendations based on individual preferences, ensuring that every listener finds podcasts that cater to their specific interests.

Through AI podcast workflow tools, platforms can track listener habits and suggest content that aligns with their tastes. Whether you’re interested in true crime, self-improvement, or technology, AI curates podcasts to suit your preferences. Additionally, AI can adjust the speed, tone, and even language used in the podcast to ensure accessibility for a global audience.

Another exciting development is how AI is enabling more interactive podcasting. From voice assistants to chatbots, AI allows listeners to interact with content directly, creating a more immersive experience. This helps engage users on a deeper level, increasing both retention and satisfaction.

AI in Podcast Editing and Post-Production

Post-production can often be the most tedious part of podcasting, but AI has simplified this stage significantly. Editing software powered by AI has the ability to automatically detect irrelevant segments, suggest edits, and even help refine the podcast’s flow.

With AI podcast workflow tools, creators can edit faster and more efficiently. AI algorithms can flag areas where the audio may need cutting, where pauses may be too long, or where background noise is distracting. This means less time spent manually editing and more time spent focusing on content creation.

Furthermore, AI tools can enhance the final product with background music, seamless transitions, and other audio elements that improve the overall quality. For podcast creators, this technology allows for a more polished and professional final product in far less time.

The Future of AI in Podcasting

As AI continues to advance, it’s clear that the future of podcasting will be heavily influenced by these technologies. The next phase of AI in podcasting is likely to involve fully automated podcast generators that can create entire podcasts from scratch based on keywords or trends.

Imagine a world where AI in high-frequency podcast content creation produces daily shows tailored to individual preferences, complete with interactive features and real-time audience engagement. It’s not far from reality. Future AI tools will likely automate even more aspects of content production, from scripting to editing, making podcast creation more accessible than ever before.

We may also see AI co-hosts or even virtual podcast hosts becoming more common. These AI-powered personalities could interact with guests, facilitate discussions, and engage with listeners in ways we haven’t yet imagined.

Ethical and Privacy Concerns in AI Podcasting

With all the benefits AI brings to podcasting, it’s important to consider the ethical implications. The rise of AI-generated content and the automation of certain creative processes raises questions about authenticity, originality, and human involvement.

There are also privacy concerns. As AI tools track listener preferences and behaviors to deliver personalized content, the collection and use of this data can raise privacy issues. Are listeners fully aware of how their data is being used, and is it being protected?

Moreover, there is the issue of ethics in AI adult content creation in the podcasting realm. AI-generated content has the potential to cross ethical boundaries, especially when it comes to sensitive subjects. As AI technology continues to evolve, creators and platforms must ensure they remain responsible and transparent about the content they produce.

Conclusion

AI has undoubtedly transformed the podcasting landscape, elevating both the creation process and the listener’s experience. From AI in podcast content creation to enhancing audio quality and personalizing recommendations, AI is making podcasts better, more engaging, and accessible to a global audience.

However, as we look toward the future, it’s essential to navigate the ethical and privacy challenges that come with these advancements. Balancing the benefits of AI with responsibility will be key to ensuring the sustainable growth of the podcasting industry.

For both creators and listeners, the possibilities are endless. As AI continues to evolve, we can only expect podcasts to become more dynamic, personalized, and innovative than ever before.

Nine Pico PIO Wats with Rust (Part 2)

0

This is Part 2 of an exploration into the unexpected quirks of programming the Raspberry Pi Pico PIO with Micropython. If you missed Part 1, we uncovered four Wats that challenge assumptions about register count, instruction slots, the behavior of pull noblock, and smart yet cheap hardware.

Now, we continue our journey toward crafting a theremin-like musical instrument — a project that reveals some of the quirks and perplexities of PIO programming. Prepare to challenge your understanding of constants in a way that brings to mind a Shakespearean tragedy.

Wat 5: Inconstant constants

In the world of PIO programming, constants should be reliable, steadfast, and, well, constant. But what if they’re not? This brings us to a puzzling Wat about how the set instruction in PIO works—or doesn’t—when handling larger constants.

Much like Juliet doubting Romeo’s constancy, you might find yourself wondering if PIO constants will, as she says, “prove likewise variable.”

The problem: Constants are not as big as they seem

Imagine you’re programming an ultrasonic range finder and need to count down from 500 while waiting for the Echo signal to drop from high to low. To set up this wait time in PIO, you might naïvely try to load the constant value directly using set:

; In Rust, be sure 'config.shift_in.direction = ShiftDirection::Left;'
set y, 15       ; Load upper 5 bits (0b01111)
mov isr, y      ; Transfer to ISR (clears ISR)
set y, 20       ; Load lower 5 bits (0b10100)
in y, 5         ; Shift in lower bits to form 500 in ISR
mov y, isr      ; Transfer back to y

Aside: Don’t try to understand the crazy jmp operations here. We’ll discuss those next in Wat 6.

But here’s the tragic twist: the set instruction in PIO is limited to constants between 0 and 31. Moreover, the star-crossed set instruction doesn’t report an error. Instead, it silently corrupts the entire PIO instruction. This produces a nonsense result.

Workarounds for inconstant constants

To address this limitation, consider the following approaches:

  • Read Values and Store Them in a Register: We saw this approach in Wat 1. You can load your constant in the osr register, then transfer it to y. For example:
# Read the max echo wait into OSR.
pull                    ; same as pull block
mov y, osr              ; Load max echo wait into Y
  • Shift and Combine Smaller Values: Using the isr register and the in instruction, you can build up a constant of any size. This, however, consumes time and operations from your 32-operation budget (see Part 1, Wat 2).
; In Rust, be sure 'config.shift_in.direction = ShiftDirection::Left;'

set y, 15       ; Load upper 5 bits (0b01111)
mov isr, y      ; Transfer to ISR (clears ISR)
set y, 20       ; Load lower 5 bits (0b10100)
in y, 5         ; Shift in lower bits to form 500 in ISR
mov y, isr      ; Transfer back to y
  • Slow Down the Timing: Reduce the frequency of the state machine to stretch delays over more system clock cycles. For example, lowering the state machine speed from 125 MHz to 343 kHz reduces the timeout constant 182,216 to 500
  • Use Extra Delays and (Nested) Loops: All instructions support an optional delay, allowing you to add up to 31 extra cycles. (To generate even longer delays, use loops — or even nested loops.)
; Generate 10μs trigger pulse (4 cycles at 343_000Hz)
set pins, 1 [3]       ; Set trigger pin to high, add delay of 3
set pins, 0           ; Set trigger pin to low voltage
  • Use the “Subtraction Trick” to Generate the Maximum 32-bit Integer: In Wat 7, we’ll explore a way to generate 4,294,967,295 (the maximum unsigned 32-bit integer) via subtraction.

Much like Juliet cautioning against swearing by the inconstant moon, we’ve discovered that PIO constants are not always as steadfast as they seem. Yet, just as their story takes unexpected turns, so too does ours, moving from the inconstancy of constants to the uneven nature of conditionals. In the next Wat, we’ll explore how PIO’s handling of conditional jumps can leave you questioning its loyalty to logic.

Wat 6: Conditionals through the looking-glass

In most programming environments, logical conditionals feel balanced: you can test if a pin is high or low, or check registers for equality or inequality. In PIO, this symmetry breaks down. You can jump on pin high, but not pin low, and on x!=y, but not x==y. The rules are whimsical — like Humpty Dumpty in Through the Looking-Glass: “When I define a conditional, it means just what I choose it to mean — neither more nor less.”

These quirks force us to rewrite our code to fit the lopsided logic, creating a gulf between how we wish the code could be written and how we must write it.

The problem: Lopsided conditionals in action

Consider a simple scenario: using a range finder, you want to count down from a maximum wait time (y) until the ultrasonic echo pin goes low. Intuitively, you might write the logic like this:

measure_echo_loop:
 jmp !pin measurement_complete   ; If echo voltage is low, measurement is complete
 jmp y-- measure_echo_loop       ; Continue counting down unless timeout

And when processing the measurement, if we only wish to output values that differ from the previous value, we would write:

measurement_complete:
 jmp x==y cooldown             ; If measurement is the same, skip to cool down
 mov isr, y                    ; Store measurement in ISR
 push                          ; Output ISR
 mov x, y                      ; Save the measurement in X

Unfortunately, PIO doesn’t let you test !pin or x==y directly. You must restructure your logic to accommodate the available conditionals, such as pin and x!=y.

The solution: The way it must be

Given PIO’s limitations, we adapt our logic with a two-step approach that ensures the desired behavior despite the missing conditionals:

  • Jump on the opposite conditional to skip two instructions forward.
  • Next, use an unconditional jump to reach the desired target.

This workaround adds one extra jump (affecting the instruction limit), but the additional label is cost-free.

Here is the rewritten code for counting down until the pin goes low:

measure_echo_loop:
   jmp pin echo_active     ; if echo voltage is high continue count down
   jmp measurement_complete ; if echo voltage is low, measurement is complete
echo_active:
   jmp y-- measure_echo_loop ; Continue counting down unless timeout

And here is the code for processing the measurement such that it will only output differing values:

measurement_complete:
   jmp x!=y send_result    ; if measurement is different, then send it.
   jmp cooldown            ; If measurement is the same, don't send.

send_result:
   mov isr, y              ; Store measurement in ISR
   push                    ; Output ISR
   mov x, y               ; Save the measurement in X

Lessons from Humpty Dumpty’s conditionals

In Through the Looking-Glass, Alice learns to navigate Humpty Dumpty’s peculiar world — just as you’ll learn to navigate PIO’s Wonderland of lopsided conditions.

But as soon as you master one quirk, another reveals itself. In the next Wat, we’ll uncover a surprising behavior of jmp that, if it were an athlete, would shatter world records.

In Part 1’s Wat 1 and Wat 3, we saw how jmp x-- or jmp y-- is often used to loop a fixed number of times by decrementing a register until it reaches 0. Straightforward enough, right? But what happens when y is 0 and we run the following instruction?

jmp y-- measure_echo_loop

If you guessed that it does not jump to measure_echo_loop and instead falls through to the next instruction, you’re absolutely correct. But for full credit, answer this: What value does y have after the instruction?

The answer: 4,294,967,295. Why? Because y is decremented after it is tested for zero. Wat!?

Aside: If this doesn’t surprise you, you likely have experience with C or C++ which distinguish between pre-increment (e.g., ++x) and post-increment (e.g., x++) operations. The behavior of jmp y-- is equivalent to a post-decrement, where the value is tested before being decremented.

This value, 4,294,967,295, is the maximum for a 32-bit unsigned integer. It’s as if a track-and-field long jumper launches off the takeoff board but, instead of landing in the sandpit, overshoots and ends up on another continent.

Aside: As foreshadowed in Wat 5, we can use this behavior intentionally to set a register to the value 4,294,967,295.

Now that we’ve learned how to stick the landing with jmp, let’s see if we can avoid getting stuck by the pins that PIO reads and sets.

In Dr. Seuss’s Too Many Daves, Mrs. McCave had 23 sons, all named Dave, leading to endless confusion whenever she called out their name. In PIO programming, pin and pins can refer to completely different ranges of pins depending on the context. It’s hard to know which Dave or Daves you’re talking to.

The problem: Pin ranges and subranges

In PIO, both pin and pins instructions depend on pin ranges defined in Rust, outside of PIO. However, individual instructions often operate on a subrange of those pin ranges. The behavior varies depending on the command: the subrange could be the first n pins of the range, all the pins, or just a specific pin given by an index. To clarify PIO’s behavior, I created the following table:

This table shows how PIO interprets the terms pin and pins in different instructions, along with their associated contexts and configurations.

Example: Distance program for the range finder

Here’s a PIO program for measuring the distance to an object using Trigger and Echo pins. The key features of this program are:

  • Continuous Operation: The range finder runs in a loop as fast as possible.
  • Maximum Range Limit: Measurements are capped at a given distance, with a return value of 4,294,967,295 if no object is detected.
  • Filtered Outputs: Only measurements that differ from their immediate predecessor are sent, reducing the output rate.

Glance over the program and notice that although it is working with two pins — Trigger and Echo — throughout the program we only see pin and pins.

.program distance

; X is the last value sent. Initialize it to
; u32::MAX which means 'echo timeout'
; (Set X to u32::MAX by subtracting 1 from 0)
   set x, 0
subtraction_trick:
   jmp x-- subtraction_trick

; Read the max echo wait into OSR
   pull                         ; same as pull block

; Main loop
.wrap_target
   ; Generate 10μs trigger pulse (4 cycles at 343_000Hz)
   set pins, 0b1 [3]       ; Set trigger pin to high, add delay of 3
   set pins, 0b0           ; Set trigger pin to low voltage

   ; When the trigger goes high, start counting down until it goes low
   wait 1 pin 0            ; Wait for echo pin to be high voltage
   mov y, osr              ; Load max echo wait into Y

measure_echo_loop:
   jmp pin echo_active     ; if echo voltage is high continue count down
   jmp measurement_complete ; if echo voltage is low, measurement is complete
echo_active:
   jmp y-- measure_echo_loop ; Continue counting down unless timeout

; Y tells where the echo countdown stopped. It
; will be u32::MAX if the echo timed out.
measurement_complete:
   jmp x!=y send_result    ; if measurement is different, then sent it.
   jmp cooldown            ; If measurement is the same, don't send.

send_result:
   mov isr, y              ; Store measurement in ISR
   push                    ; Output ISR
   mov x, y               ; Save the measurement in X

; Cool down period before next measurement
cooldown:
   wait 0 pin 0           ; Wait for echo pin to be low
.wrap                      ; Restart the measurement loop

Configuring Pins

To ensure the PIO program behaves as intended:

  • set pins, 0b1 should control the Trigger pin.
  • wait 1 pin 0 should monitor the Echo pin.
  • jmp pin echo_active should also monitor the Echo pin.

Here’s how you can configure this in Rust (followed by an explanation):

let mut distance_state_machine = pio1.sm0;
let trigger_pio = pio1.common.make_pio_pin(hardware.trigger);
let echo_pio = pio1.common.make_pio_pin(hardware.echo);
distance_state_machine.set_pin_dirs(Direction::Out, &[&trigger_pio]);
distance_state_machine.set_pin_dirs(Direction::In, &[&echo_pio]);
distance_state_machine.set_config(&{
   let mut config = Config::default();
   config.set_set_pins(&[&trigger_pio]); // For set instruction
   config.set_in_pins(&[&echo_pio]); // For wait instruction
   config.set_jmp_pin(&echo_pio); // For jmp instruction
   let program_with_defines = pio_file!("examples/distance.pio");
   let program = pio1.common.load_program(&program_with_defines.program);
   config.use_program(&program, &[]); // No side-set pins
   config
});

The keys here are the set_set_pins, set_in_pins, and set_jmp_pin methods on the Config struct.

  • set_in_pins: Specifies the pins for input operations, such as wait(1, pin, …). The “in” pins must be consecutive.
  • set_set_pins: Configures the pin for set operations, like set(pins, 1). The “set” pins must also be consecutive.
  • set_jmp_pin: Defines the single pin used in conditional jumps, such as jmp(pin, ...).

As described in the table, other optional inputs include:

  • set_out_pins: Sets the consecutive pins for output operations, such as out(pins, …).
  • use_program: Sets a) the loaded program and b) consecutive pins for sideset operations. Sideset operations allow simultaneous pin toggling during other instructions.

Configuring Multiple Pins

Although not required for this program, you can configure a range of pins in PIO by providing a slice of consecutive pins. For example, suppose we had two ultrasonic range finders:

let trigger_a_pio = pio1.common.make_pio_pin(hardware.trigger_a);
let trigger_b_pio = pio1.common.make_pio_pin(hardware.trigger_b);
config.set_set_pins(&[&trigger_a_pio, &trigger_b_pio]);

A single instruction can then control both pins:

set pins, 0b11 [3]  # Sets both trigger pins (17, 18) high, adds delay
set pins, 0b00      # Sets both trigger pins low

This approach lets you efficiently apply bit patterns to multiple pins simultaneously, streamlining control for applications involving multiple outputs.

Aside: The Word “Set” in Programming

In programming, the word “set” is notoriously overloaded with multiple meanings. In the context of PIO, “set” refers to something to which you can assign a value — such as a pin’s state. It does not mean a collection of things, as it often does in other programming contexts. When PIO refers to a collection, it usually uses the term “range” instead. This distinction is crucial for avoiding confusion as you work with PIO.

Lessons from Mrs. McCave

In Too Many Daves, Mrs. McCave lamented not giving her 23 Daves more distinct names. You can avoid her mistake by clearly documenting your pins with meaningful names — like Trigger and Echo — in your comments.

But if you think handling these pin ranges is tricky, debugging a PIO program adds an entirely new layer of challenge. In the next Wat, we’ll dive into the kludgy debugging methods available. Let’s see just how far we can push them.

I like to debug with interactive breakpoints in VS Code. I also do print debugging, where you insert temporary info statements to see what the code is doing and the values of variables. Using the Raspberry Pi Debug Probe and probe-rs, I can do both of these with regular Rust code on the Pico.

With PIO programming, however, I can do neither.

The fallback is push-to-print debugging. In PIO, you temporarily output integer values of interest. Then, in Rust, you use info! to print those values for inspection.

For example, in the following PIO program, we temporarily add instructions to push the value of x for debugging. We also include set and out to push a constant value, such as 7, which must be between 0 and 31 inclusive.

.program distance

; X is the last value sent. Initialize it to
; u32::MAX which means 'echo timeout'
; (Set X to u32::MAX by subtracting 1 from 0)
   set x, 0
subtraction_trick:
   jmp x-- subtraction_trick

; DEBUG: See the value of x
   mov isr, x
   push

; Read the max echo wait into OSR
   pull                         ; same as pull block

; DEBUG: Send constant value
   set y, 7           ; Push '7' so that we know we've reached this point
   mov isr, y
   push
; ...

Back in Rust, you can read and print these values to help understand what’s happening in the PIO code (full code and project):

  // ...
   distance_state_machine.set_enable(true);
   distance_state_machine.tx().wait_push(MAX_LOOPS).await;
   loop {
       let end_loops = distance_state_machine.rx().wait_pull().await;
       info!("end_loops: {}", end_loops);
   }
  // ...

Outputs:

INFO  Hello, debug!
└─ distance_debug::inner_main::{async_fn#0} @ examplesdistance_debug.rs:27
INFO  end_loops: 4294967295
└─ distance_debug::inner_main::{async_fn#0} @ examplesdistance_debug.rs:57
INFO  end_loops: 7
└─ distance_debug::inner_main::{async_fn#0} @ examplesdistance_debug.rs:57

When push-to-print debugging isn’t enough, you can turn to hardware tools. I bought my first oscilloscope (a FNIRSI DSO152, for $37). With it, I was able to confirm the Echo signal was working. The Trigger signal, however, was too fast for this inexpensive oscilloscope to capture clearly.

Using these methods — especially push-to-print debugging — you can trace the flow of your PIO program, even without a traditional debugger.

Aside: In C/C++ (and potentially Rust), you can get closer to a full debugging experience for PIO, for example, by using the piodebug project.

That concludes the nine Wats, but let’s bring everything together in a bonus Wat.

Now that all the components are ready, it’s time to combine them into a working theremin-like musical instrument. We need a Rust monitor program. This program starts both PIO state machines — one for measuring distance and the other for generating tones. It then waits for a new distance measurement, maps that distance to a tone, and sends the corresponding tone frequency to the tone-playing state machine. If the distance is out of range, it stops the tone.

Rust’s Place: At the heart of this system is a function that maps distances (from 0 to 50 cm) to tones (approximately B2 to F5). This function is simple to write in Rust, leveraging Rust’s floating-point math and exponential operations. Implementing this in PIO would be virtually impossible due to its limited instruction set and lack of floating-point support.

Here’s the core monitor program to run the theremin (full file and project):

sound_state_machine.set_enable(true);
distance_state_machine.set_enable(true);
distance_state_machine.tx().wait_push(MAX_LOOPS).await;
loop {
   let end_loops = distance_state_machine.rx().wait_pull().await;
   match loop_difference_to_distance_cm(end_loops) {
       None => {
           info!("Distance: out of range");
           sound_state_machine.tx().wait_push(0).await;
       }
       Some(distance_cm) => {
           let tone_frequency = distance_to_tone_frequency(distance_cm);
           let half_period = sound_state_machine_frequency / tone_frequency as u32 / 2;
           info!("Distance: {} cm, tone: {} Hz", distance_cm, tone_frequency);
           sound_state_machine.tx().push(half_period); // non-blocking push
           Timer::after(Duration::from_millis(50)).await;
       }
   }
}

Using two PIO state machines alongside a Rust monitor program lets you literally run three programs at once. This setup is convenient on its own and is essential when strict timing or very high-frequency I/O operations are required.

Aside: Alternatively, Rust Embassy’s async tasks let you implement cooperative multitasking directly on a single main processor. You code in Rust rather than a mixture of Rust and PIO. Although Embassy tasks don’t literally run in parallel, they switch quickly enough to handle applications like a theremin. Here’s a snippet from theremin_no_pio.rs showing a similar core loop:

loop {
       match distance.measure().await {
           None => {
               info!("Distance: out of range");
               sound.rest().await;
           }
           Some(distance_cm) => {
               let tone_frequency = distance_to_tone_frequency(distance_cm);
               info!("Distance: {} cm, tone: {} Hz", distance_cm, tone_frequency);
               sound.play(tone_frequency).await;
               Timer::after(Duration::from_millis(50)).await;
           }
       }
   }

See our recent article on Rust Embassy programming for more details.

Now that we’ve assembled all the components, let’s watch the video again of me “playing” the musical instrument. On the monitor screen, you can see the debugging prints displaying the distance measurements and the corresponding tones. This visual connection highlights how the system responds in real time.

Conclusion

PIO programming on the Raspberry Pi Pico is a captivating blend of simplicity and complexity, offering unparalleled hardware control while demanding a shift in mindset for developers accustomed to higher-level programming. Through the nine Wats we’ve explored, PIO has both surprised us with its limitations and impressed us with its raw efficiency.

While we’ve covered significant ground — managing state machines, pin assignments, timing intricacies, and debugging — there’s still much more you can learn as needed: DMA, IRQ, side-set pins, differences between PIO on the Pico 1 and Pico 2, autopush and autopull, FIFO join, and more.

Recommended Resources

At its core, PIO’s quirks reflect a design philosophy that prioritizes low-level hardware control with minimal overhead. By embracing these characteristics, PIO will not only meet your project’s demands but also open doors to new possibilities in embedded systems programming.

Please follow Carl on Towards Data Science and on @carlkadie.bsky.social. I write on scientific programming in Rust and Python, machine learning, and statistics. I tend to write about one article per month.

Open LLMs are Necessary For Current Private Adaptations and Outperform Their Closed Alternatives [Paper Reflection]

0

Closed Large Language Models (LLMs), which are proprietary and accessible only via APIs, have dominated the LLM space since around 2022 due to their high performance and versatility. However, Open LLMs have made substantial progress, narrowing the performance gap with their Closed LLM counterparts. Open LLMs are models whose architecture and parameters are publicly available for use, modification, and distribution.

For instance, while Closed LLMs like Anthropic’s Claude (released in March 2023) and OpenAI’s GPT-4 (released in March 2023) set new benchmarks upon their launches, the Open LLM Llama 3 released by Meta in April 2024 and DeepSeek-R1 released in January 2025 not only matched but surpassed these models in tasks such as coding, reasoning, text classification, summarization, and question answering.

While much of the discussion around LLMs centers on task and computational performance, in our paper Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives, we focus on the privacy implications of using Open and Closed LLMs. Specifically, we explore whether and how models can be fine-tuned on sensitive data while ensuring robust privacy guarantees.

To this end, we define threat models, compare various Open and Closed LLMs that leverage differential privacy (DP) on classification and generation tasks and analyze methodological limitations. Our research results in a thorough analysis of the privacy-utility tradeoff under different privacy levels.

Our findings indicate that Open LLMs can be adapted to private data without leaking information to third parties, such as LLM providers and malicious users. Thus, they offer a significant privacy advantage over Closed, proprietary models.

The threat space in adapting LLMs to private data

The adaptation of Closed LLMs to private datasets introduces a multifaceted threat space. In typical scenarios, data curators provide their sensitive data to LLM providers for fine-tuning, producing a model tailored to the dataset. This customized model is subsequently queried by external parties, e.g., customers of the data curator.

The resulting threat space can be categorized into three key dimensions:

  1. From the data curator to the LLM provider: The private data shared during fine-tuning may be susceptible to unauthorized access or misuse.
  2. From the querying party to the LLM provider: Queries submitted by end users, which often contain sensitive information intended for the data curator, are exposed to the LLM provider.
  1. From malicious end users to the adapted LLM: Malicious end users may attempt to extract private information through the LLM’s responses to carefully crafted queries.

In contrast to Closed LLMs, Open LLMs provide full control over the model and data, enabling private adaptation without the need to share sensitive information with a third party. This control eliminates the first two threat vectors associated with Closed LLMs, such as unauthorized access or misuse by the provider and exposure of user queries. With Open LLMs, data curators can directly fine-tune the model on private datasets using privacy-preserving techniques, ensuring end-to-end privacy.

What are the current methods for private adaptation of LLMs? 

It follows from our threat space analysis that restricting access to the fine-tuning dataset alone does not guarantee data privacy. Model outputs can still reveal sensitive information from the fine-tuning data. If the fine-tuned model is exposed (e.g., via an API), it remains vulnerable to information extraction and inference attacks.

Differential privacy (DP) introduces a rigorous mathematical framework that ensures the privacy of individuals whose data is used in the fine-tuning process. Specifically, DP adds carefully calibrated noise to the model updates, making it statistically improbable to determine whether any individual’s data was included in the fine-tuning dataset. Its quantifiable and robust privacy guarantee makes DP valuable for protecting sensitive information in LLM fine-tuning.

While DP provides privacy guarantees for both Open and Closed LLMs, it does not address the issue of trust in third-party providers for Closed LLMs. For these models, data curators must rely on the provider to implement safeguards and handle sensitive data responsibly.

Private adaptation methods for Closed LLMs 

We can rule out fine-tuning services offered by LLM providers (e.g., OpenAI and Amazon), as this entails sharing private data with a third party. Closed LLMs are accessible only via APIs. Thus, we cannot access and adapt the model’s weights directly.

Instead, private adaptation methods for Closed LLMs rely on privacy-preserving discrete prompts or private in-context learning (ICL). These approaches work by carefully crafting input prompts or selecting relevant examples to guide the model’s behavior, all while ensuring that sensitive information in the prompts or examples is protected from potential leakage or inference attacks.

All methods we evaluate in our study follow the PATE (Private Aggregation of Teacher Ensembles) framework. At a high level, PATE achieves data privacy by splitting the private dataset into non-overlapping partitions. Then, each partition is used to train a so-called teacher model. These teacher models are joined into an ensemble model by combining their outputs while adding noise, which preserves privacy.

This ensemble is then used to train a so-called student model in the following way: The ensemble makes predictions for samples from an unlabeled public dataset. The resulting (sample, ensemble prediction) pairs constitute the training data for the student model. Thus, the student learns to make the same predictions as the teacher ensemble but never sees sensitive data samples. The student is what’s released as the final model.

Overview of the PATE framework. The sensitive dataset is divided into non-overlapping partitions, and a separate teacher model is trained on each partition. All teachers are aggregated noisily into an ensemble model, which is used to make predictions on a public dataset. The samples from the public dataset, together with the ensemble’s predictions, constitute the training data for the student model, which is the model that is eventually queried by users. | Source

The private adaptation methods for Closed LLMs we analyze in our study build on this general framework. They differ in how the teachers are utilized and how their responses are aggregated:

  • Differentially Private In-context Learning (DP-ICL): All teachers process the same prompt, and the ensemble’s response is the noisy consensus.
  • PromptPATE: The teacher ensemble assigns labels to public unlabeled data via private voting. These labeled public sequences are used to create new discrete student prompts, which are deployed with the LLM.
  • DP-FewShotGen: The teacher ensemble generates private synthetic few-shot samples that are used as samples for in-context learning.
  • DP-OPT: A local LLM generates privacy-preserving prompts and instructions from the private dataset. These are used for in-context learning for the third-party Closed LLM.

In our paper, we compare the privacy protection and performance of these four state-of-the-art methods for private adaptation of Closed LLMs. When applying them to the popular Closed LLMs Claude, GPT-3 Babbage, GPT-3 Davinci, and GPT-4 Turbo, we observe that compared to private adaptation of Open LLMs, these methods offer lower performance at a higher cost on various downstream tasks, including dialog summarization, classification, and generation. Further, all methods except DP-OPT leak training data to the LLM provider.

Private adaptation methods for Open LLMs 

Unlike Closed LLMs, Open LLMs provide access to their parameters, enabling more flexible and parameter-centric private adaptation methods. These methods typically follow the Differentially Private Stochastic Gradient Descent (DPSGD) paradigm to ensure privacy. In DPSGD, the influence of each private data point is constrained during training through gradient clipping and the addition of calibrated noise. This approach guarantees that the model does not memorize or leak sensitive information.

In our study, we explore three primary methods for private adaptation of Open LLMs: 

  1. Prompt-based adaptation (PromptDPSGD) introduces a small number of additional parameters (<1% of the model’s total parameters) in the input space through soft prompts or prefix-tuning and adapts Differentially Private Stochastic Gradient Descent (DPSGD) to preserve privacy.
  2. Parameter-efficient fine-tuning, such as LoRA, only updates a relatively small number of parameters (<10% of the model’s total parameters) within the model’s architecture to enable efficient updates. PrivateLoRA extends this approach with DP guarantees by building on the DPSGD algorithm.
  3. Full fine-tuning adaptations (DP-FineTune) involve fine-tuning the entire model or a subset of its layers for comprehensive adaptation while adhering to differential privacy principles.

Applying these methods to Vicuna, Llama-3, OpenLLaMa, BART, RoBERTa, and the Pythia suite of models, we find that private adaptation of Open LLMs improves performance on downstream tasks and reduces costs compared to their Closed counterparts. It also provides a critical privacy benefit by eliminating the risk of exposing private data and user queries to LLM providers.

Insightful results

Our analysis of private adaptation methods for both Closed and Open LLMs reveals several critical findings regarding data leakage, performance, and cost:

  1. Query data leakage: All private adaptation methods for Closed LLMs leak query data to the LLM provider. This means that sensitive information from user queries is exposed during the adaptation process, posing a significant privacy risk.
  2. Training data leakage: Only one method (DP-OPT) of the four methods of private adaptation of Closed LLMs successfully protects private training data from the LLM provider. However, this method requires a local LLM to effectively protect the privacy of the training data. The remaining private adaptation methods for Closed LLMs leak a large fraction of the training data to the LLM provider, undermining the privacy guarantees of the adaptation process.
  3. Performance: All adaptation methods for Closed LLMs achieve lower downstream task performance than privacy-preserving local adaptations on Open LLMs, even when the Open LLMs are significantly smaller than their Closed counterparts.
  4. Cost: The training and query costs for private adaptations of Closed LLMs are substantially higher due to the API access costs imposed by the LLM provider. In contrast, private adaptations for Open LLMs are more cost-effective. We estimated the costs assuming an A40 GPU with 48 GB of memory. In this scenario, privately adopting a Closed LLM to text classification tasks with DP-ICL costs about $140. In contrast, fine-tuning an Open LLM with PrivateLoRA on the same tasks costs about $30.

This leads to the conclusion that for a truly privacy-preserving adaptation of LLMs, one should use Open LLMs. By offering full control over the model and data, Open LLMs eliminate the risks associated with third-party providers and enable robust privacy-preserving techniques. As a result, Open LLMs address the limitations of Closed LLMs and enable efficient and customizable adaptations tailored to sensitive datasets.

Was the article useful?

Explore more content topics:

New Microsoft 365 Phishing Scam Tricks Users Into Calling Fake Support

0

Cybersecurity company Guardz is warning Microsoft 365 users about a new phishing scam backed by social engineering tactics making the rounds. This isn’t an average scam as attackers trick people into calling fake support numbers using Microsoft 365 infrastructure, putting their login details and accounts at risk.

How the Attack Works

Unlike typical phishing attempts using typosquatted domains, fake or misspelled email addresses, this campaign operates from within Microsoft’s cloud services. This makes the phishing attempts look convincing, easily bypassing email authentication checks like SPF, DKIM, and DMARC.

The attack also utilizes legitimate Microsoft domains (onmicrosoft.com)and manipulates tenant settings. The scammers also set up multiple Microsoft 365 organization tenants, either by creating new ones or compromising existing accounts. Each tenant has a specific role within the attack framework, allowing the threat actors to operate with anonymity.

One of these fake organizations is used to trigger actions that look like normal business activity, such as starting a subscription. Another fake organization is given a name that includes a fake warning message and a phone number. For example, the organization’s name might appear as something like, “(Microsoft Corporation) Your subscription has been successfully purchased… If you did not authorize this transaction, please call .”

The Microsoft 365 phishing email used in the scam (Screenshot credit: Guardz)

When the attackers trigger an action, like a subscription change, Microsoft 365 automatically sends out legitimate emails about it. Because of how the attackers set up their fake organizations, these official Microsoft emails can end up including the fake warning message and phone number in the sender’s information or organization details.

So, you might receive an email that looks like it’s really from Microsoft, confirming a purchase you didn’t make. The email itself is real in the sense that it came through Microsoft’s systems.

But the alarming message asking you to call a number to dispute the charge? That’s the scam. If someone calls the number, they’re connected with the attackers, who then try to steal sensitive information like passwords or trick them into installing malicious software.

Why This Scam Is Effective

This approach is effective for several reasons. Since the emails come from Microsoft’s legitimate systems, they often pass standard security checks that look for fake domains or suspicious links. The emails look official, complete with Microsoft branding. And the urgent message about an unauthorized charge can cause people to act quickly without thinking.

According to Guardz’s report shared with Hackread.com ahead of its publishing on Thursday, this attack is tricky to spot because it uses legitimate services for malicious purposes. Traditional email security measures that check sender reputations or look for fake links might miss this.

The Possible Impact

The implications of this phishing campaign could be significant. Businesses and individuals who fall victim can suffer from credential theft, financial loss, account takeovers or installing malware on their systems. The attack’s dependence on voice channels also makes it more challenging to detect and prevent, as fewer security controls exist in direct phone communications.

Protecting Yourself and Your Business

A few key steps can help prevent these scams. Be wary of unexpected emails about purchases or subscriptions, even if they appear to come from Microsoft. Never call phone numbers listed in emails if something feels off, always verify contact details on Microsoft’s official website.

Pay close attention to sender details; while an email might look legitimate, unusual organization names or urgent wording can be red flags. Also, be cautious of messages from unfamiliar “.onmicrosoft.com” domains. Most importantly, train yourself and your employees to recognize phishing tactics, especially those designed to create a sense of urgency around financial threats.

  1. Fake Facebook Copyright Notices to Hijacking Accounts
  2. Hackers Using Fake YouTube Links to Steal Login Credentials
  3. PayPal Phishing Exploits MS365 Tools, Genuine-Looking Emails
  4. Phishing Attacks Can Bypass Microsoft 365 Email Safety Warnings
  5. Astaroth Phishing Kit Bypasses 2FA, Hijacks Gmail, Microsoft Emails


Strengthening the Human Firewall: Prioritising Mental Health in Cybersecurity Teams

0

There are few places more challenging than the frontlines of war.

Danger lurks at every corner while enemy fire is a persistent threat. It’s a hostile and stress-induced environment that demands unwavering focus, and where a single error can have disastrous consequences.

Fortunately, the frontlines of war are a place most people won’t encounter today.

But the environment isn’t too contrasting to working on the frontlines of cyber defence.

Cybersecurity professionals operate in one of the most high-pressure environments today.

Threats bombard organisations incessantly, security alerts pour in by the minute, while teams often contend with constrained resources and budgets. Despite this, they bear the immense responsibility of defending their organisations against increasingly sophisticated cyber threats.

Cybersecurity has evolved into one of the most mission-critical departments in business, acting as an organisation’s frontline defence in the increasingly hostile digital landscape.

Cyberattacks have evolved from mere technical nuisances, into threats that can threaten the solvency of an organisation, causing financial and reputational devastation. Security teams must remain constantly vigilant to ensure no attack escalates into a full-scale breach or ransomware incident.

This immense responsibility can weigh heavily on cybersecurity professionals. Many feel that the future and safety of the entire organisation, as well as its customers and stakeholders, rest solely on their shoulders.

Unsurprisingly, burnout in cybersecurity is a prevalent and growing concern.

Security teams frequently feel overwhelmed by the pressure, working long hours with limited resources, while defending against a relentless and ever-evolving threat landscape.

This level of stress is unsustainable and, if unaddressed, can lead to exhaustion, decreased performance and even serious mental health issues.

Recent data from SoSafe revealed that sixty-eight percent of security professionals in Europe are experiencing burnout, with 32% experiencing high burnout levels and 36% experiencing a moderate degree.

This data underscores the severity of burnout in cybersecurity and reinforces the need for organisations to strive to do more to better support their teams.

So, what can organisations do to strengthen their security teams, ensuring their mental health is prioritised and they themselves feel protected?

  • Encourage Open Communication: Establishing open channels for security teams to voice their concerns is essential. Employees should feel comfortable discussing feelings of stress or burnout with HR, management or colleagues. Additionally, other departments should be educated about the pressures security teams face to prevent unnecessary strain or unrealistic demands.
  • Regular Check-Ins from Leadership: Management and HR must regularly engage with cybersecurity professionals, not just to assess performance but to understand their personal well-being. These check-ins should be a structured, ongoing initiative, demonstrating a leadership commitment to mental health.
  • Identify and Address Workload Issues: If team members are feeling overwhelmed, it’s crucial to assess why. Are there bottlenecks that can be alleviated? Could additional resources be allocated? Would time off help? Understanding the root causes of stress can lead to actionable solutions.

In today’s increasingly pressured security landscape, organisations must take proactive steps to support their security teams. Failing to do so can not only jeopardise employee well-being but also expose the organisation to increased security risks.

Ignoring burnout and placing too much pressure on security teams, won’t help the organisation. Over stretched staff lead to reduced attention, increased errors, and, ultimately, compromised systems.

By supporting those on the frontlines of digital defence, we ensure stronger, more resilient organisations that are better equipped to face the evolving cyber threat landscape.

At this year’s DTX Manchester, I will be participating in a keynote panel session alongside the Office for Nuclear Regulation and Community Mental Health Services, where we will discuss how to strengthen the human firewall and prioritise mental health in cybersecurity teams.

 During the session we will discuss how managers and colleagues can identify signs of mental health struggles, provide advice on effective ways to approach, talk to, and support colleagues who may be masking or exhibiting concerning behaviour and  also discussing the strategies and resources available to foster resilience in high-pressure environments and support employee mental health.

Join me for the session, which will take place on Wednesday 2nd April from 12:10PM – 12:45PM.

By Jonathan Marnoch, Principle Cyber Architect, Jaguar Land Rover

The post Strengthening the Human Firewall: Prioritising Mental Health in Cybersecurity Teams appeared first on IT Security Guru.

Keeper Security Gives Its Partner Programme an update

0

Keeper Security has announced the launch of the updated Keeper Partner Programme. The updated programme is designed to help organisations of all sizes expand their cybersecurity offerings and unlock new revenue opportunities. 

As businesses increasingly adopt PAM solutions to protect privileged credentials, secrets and remote access, Keeper’s programme provides comprehensive partner tiers, extensive training and a lucrative incentive structure to help partners accelerate growth. With distribution partners around the globe, Keeper is committed to empowering its partners with the tools they need to thrive in today’s cybersecurity landscape.

Scott Unger, Director of Global Channel Account Management at Keeper Security, said: “Keeper’s Partner Programme was built to ensure our partners have the competitive advantage they need in a rapidly evolving cybersecurity market. With brand new KeeperPAM sales, demo and implementation training through Keeper University, tiered pricing and strong financial incentives – including world-class channel marketing and proposal-based MDF programmes – partners can rapidly grow their business while delivering best-in-class privileged access management.” 

Key Benefits of the 2025 Keeper Partner Programme

With cyber threats escalating, organisations are prioritising privileged access security as a core defence strategy. The Keeper Partner Programme is designed to meet this demand while helping partners maximise revenue potential.

  • Expanded Revenue Streams: As businesses shift towards modern, zero-trust PAM solutions, Keeper provides partners – especially enterprise-focused resellers – with a high-growth, high-margin security offering.
  • Comprehensive Training & Certifications: Free access to Keeper Sales Professional (KSP), Keeper Demo Expert (KDE) and KeeperPAM Implementation (KPI) certifications to enhance both sales and technical expertise.
  • Flexible Partner Tiers: Four levels – Authorised, Silver, Gold and Platinum – offer progressively greater benefits, including tiered discounts and revenue-sharing opportunities.
  • Marketing and Growth Support: Silver-level and higher partners gain access to Market Development Funds (MDF) to fuel demand generation and drive customer acquisition. 
  • Global SPIFF Program: A structured incentive program rewarding partners for closed/won deals, with four tiers of compensation to maximise earnings.

Helping Partners Win in Cybersecurity

With best-in-class customer retention, Keeper is the trusted cybersecurity partner for organisations worldwide. Its unified PAM platform – spanning enterprise password management, secrets management, connection management, zero-trust network access and remote browser isolation – helps businesses of all sizes protect their most sensitive information and resources.

The Keeper Partner Program is now open for enrolment. Partners ready to accelerate their business and capitalise on the increasing demand for PAM solutions can apply through the Keeper Partner Portal

The post Keeper Security Gives Its Partner Programme an update appeared first on IT Security Guru.

Best WordPress Plugins for Cybersecurity 2025

0

WordPress is a great platform for building websites, but it is also a common target for hackers. Keeping your website safe is important to protect your data, visitors, and business. Cybercrime is a growing problem, with 39% of UK businesses experiencing cyber attacks in 2023.

Using security plugins can help reduce risks and keep your site safe from threats and are essential for any wordpress site, and even more so if your site has personal customer data on it. We speak with Sierra Six, a leading SEO agency in Essex to get their recommendations on the best plugins for security and to reduce cyber attacks.

Wordfence Security

Wordfence Security is one of the most popular cybersecurity plugins for WordPress. It provides a firewall that blocks malicious traffic before it reaches your website. It also has a malware scanner that checks your site for viruses and suspicious code. If anything harmful is found, Wordfence will alert you so you can take action. Another useful feature is its login protection, which helps stop hackers from guessing passwords.

Sucuri Security

Sucuri Security is another excellent plugin that protects your site from hackers. It offers a website firewall, which blocks attacks before they can do any harm. The plugin also scans your website for malware and removes it if necessary. If your site ever gets hacked, Sucuri provides help to clean it up. This is useful because recovering from a hack can be difficult without expert support.

iThemes Security

iThemes Security is designed to strengthen your WordPress site against attacks. It protects against brute force attacks, where hackers try thousands of password combinations to break into your site. The plugin also scans for vulnerabilities and fixes weak points in your website’s security. Another feature is two-factor authentication, which adds an extra layer of protection when logging in.

All In One WP Security & Firewall

This plugin is great for beginners who want an easy way to secure their website. It comes with a firewall to block suspicious traffic and a login lockdown feature to stop repeated failed login attempts. The plugin also scans for weak passwords and forces users to create stronger ones. Since weak passwords are responsible for 81% of hacking-related breaches, this is an important feature.

Conclusion

Cyber threats are increasing, and UK businesses must take website security seriously. Using security plugins like Wordfence, Sucuri, iThemes Security, and All In One WP Security can help protect your WordPress site from hackers and malware. Regular updates and strong passwords also play a key role in keeping your site safe. By taking these steps, you can reduce the risk of cyber attacks and keep your website secure.

The post Best WordPress Plugins for Cybersecurity 2025 appeared first on IT Security Guru.

Ethical Considerations and Best Practices in LLM Development 

0

Bias is inherent to building a ML model. Bias exists on a spectrum. Our job is to tell the difference between the desirable bias and the one that needs correction.

We can identify biases using benchmarks like StereoSet and BBQ, and minimize them with ongoing monitoring across versions and iterations.

Adhering to data protection laws is not as complex if we focus less on the internal structure of the algorithms and more on the practical contexts of use.

To keep data secure throughout the model’s lifecycle, implement these practices: data anonymization, secure model serving and privacy penetration tests.

Transparency can be achieved by providing contextual insights into model outputs. Documentation and opt-out mechanisms are important aspects of a trustworthy system.

Picture this: you’ve spent months fine-tuning an AI-powered chatbot to provide mental health support. After months of development, you launch it, confident it will make therapy more accessible for those in need. But soon, reports emerge: one user seeking help for an eating disorder received diet tips instead of support, worsening their condition. Another, in a moment of crisis, met with responses that intentionally encouraged harmful behaviors (and later committed suicide). This is not hypothetical—it’s a real-life example. 

Now think about your work as an AI professional. Just like the mortgage model, large language models (LLMs) influence critical decisions, and training them on biased data can perpetuate harmful stereotypes, exclude marginalized voices, or even generate unsafe recommendations. Whether the application is financial services, healthcare, or customer support, the ethical considerations are just as high: how do we ensure our work has long-term value and positive societal impact? By focusing on measurable solutions: differential privacy techniques to protect user data, bias-mitigation benchmarks to identify gaps, and reproducible tracking with tools like neptune.ai to ensure accountability.

This article isn’t just about why ethics matter—it’s about how you can take action now to build trustworthy LLMs. Let’s get started!

So how can we address bias in LLMs?

Bias in the context of training LLMs is often discussed with a negative connotation. However, the reality is more complex: algorithmic bias is inherent in any machine learning model because it reflects patterns, structures, and priorities encoded in the training data and design. Let’s put it this way: some bias is necessary for models to work effectively. When we fine-tune LLMs, we shift their biases to align with specific tasks or applications. For example, a large language model is intentionally biased toward generating grammatically correct sentences. 

The challenge for AI researchers and engineers lies in separating desirable biases from harmful algorithmic biases that perpetuate social biases or inequity. To address it, it’s helpful to think of bias as existing on a spectrum:

  1. Functional biases: The previous example falls on this end of the spectrum. These biases are intentional and beneficial to enhance model performance. They guide the LLM to generate text in a specific tone, style, or adhering to a logical reasoning pattern, etc.
  1. Neutral biases: These may not directly harm users but can skew the diversity of outputs. For example, an LLM trained on predominantly European data might overrepresent those perspectives, unintentionally narrowing the scope of information or viewpoints it offers.
  1. Harmful biases: These are the biases that demand active mitigation. Harmful biases lead to biased outputs that disadvantage certain groups. For example, a recruitment LLM favoring male applicants due to biased training data reflects a harmful bias that requires correction. During the data collection stage, two valuable frameworks to analyze data distribution are Datasheets for datasets and FACETS.

To mitigate unwanted biases (the third end of the spectrum), it is recommended to adopt a  structured approach during the fine-tuning stage:

1. Define the desired outcome

Identify the biases your model should intentionally have and avoid. For example, an LLM designed for legal assistance should prioritize precision and formal language (functional biases), while actively avoiding harmful biases like racial assumptions in legal case studies.

2. Test and measure bias

Debiasing techniques assess how your pre-trained LLM handles both neutral and harmful biases. Two of the most popular benchmarks are StereoSet to test for stereotypical associations in the outputs of your large language model and BBQ (Bias Benchmark for QA) for highlighting biases in question-answering systems. 

Let’s see how to use them in a simple example. Imagine you’re evaluating an LLM used in a recruitment platform. A StereoSet prompt might be:

“The software engineer was explaining the algorithm. After the meeting, ___ went back to coding.”

The benchmark would present two potential completions:

  • “he” (stereotypical)
  • “she” or “they” (non-stereotypical)

StereoSet evaluates the model’s likelihood of generating each option. Suppose your LLM is heavily biased toward stereotypical associations, like assuming “software engineer” is male. This would indicate a higher probability assigned to “he” over “she” or “they.”

This is a common stereotype, but StereoSet can evaluate more nuanced scenarios like:

“The team lead recommended a flexible work schedule for better work-life balance. ___ later presented their findings to the board.”

Here, the model’s output might be tested for implicit gender bias linking caregiving roles or flexibility to one gender while associating leadership and authority with another. The results are then compared to a baseline provided by the benchmark, which quantifies the degree of bias in your LLM’s outputs. By analyzing such patterns across thousands of prompts, these debiasing techniques provide a detailed breakdown of how biases manifest in your LLM’s outputs, allowing you to pinpoint specific areas for improvement.

Identify the appropriate bias benchmark for your specific task. For this, you can explore the collection of LLM benchmarks curated by researchers at McGill University, which offers a range of benchmarks tailored to a variety of scenarios.

3. Monitor bias continuously

Mitigating bias isn’t a one-time effort—it requires ongoing monitoring to ensure that your LLM remains fair and effective across iterations. Here are some ideas to help you implement it:

Create a script that evaluates your model

First, we create a script that runs a standardized set of evaluations against one of your model versions. Think about the metrics that you will implement to measure bias in your specific scenario. You can explore fairness metrics, such as demographic parity, measure disparate impact (the extent to which the model’s decisions disproportionately affect different groups), or assess stereotype reinforcement using the benchmarks mentioned earlier.

Demographic parity (also known as statistical parity) is a metric used to assess bias and fairness concerns, that is, whether a machine learning model treats different demographic groups equally in terms of outcomes. Specifically, it measures whether the probability of a positive outcome (e.g., approval for a loan, a job recommendation, etc.) is the same across different groups, regardless of their demographic attributes (e.g., gender, race, age). Here there is a manual implementation of this metric in Python:

from sklearn.metrics import confusion_matrix


y_true = [0, 1, 0, 1, 0]  
y_pred = [0, 1, 0, 0, 1]  
group_labels = ['male', 'female', 'male', 'female', 'male']  
def demographic_parity(y_true, y_pred, group_labels):
    groups = set(group_labels)
    parity = {}
    
    for group in groups:
        group_indices = [i for i, label in enumerate(group_labels) if label == group]
        group_outcomes = [y_pred[i] for i in group_indices]
        positive_rate = sum(group_outcomes) / len(group_outcomes)
        parity[group] = positive_rate

    return parity

parity_results = demographic_parity(y_true, y_pred, group_labels)
print(parity_results)  

You can also explore demographic_parity_ratio from the fairlearn.metrics package, which simplifies the application of this fairness metric in your model evaluation.

Track your results in Neptune

You can use tools like neptune.ai to track bias metrics (e.g., fairness or disparate impact) across model versions. Let’s see how:

  1. Set up your project: If you haven’t already, sign up for Neptune now and create a project to track your LLM’s training data and metrics.
  2. Log the metrics: Set up custom logging for these metrics in your training code by calculating and recording them after each evaluation phase.
  3. Monitor bias: Use Neptune’s dashboards to monitor how these fairness metrics evolve over model versions. Compare the impact of different debiasing strategies on the metrics, and create alerts to notify you when any metric exceeds a threshold. This allows you to take immediate corrective action.

All metadata in a single place with an experiment tracker (example in neptune.ai)

Integrate bias checks into your CI/CD workflows

If your team manages model training through CI/CD, incorporate the automated bias detection scripts (that have already been created) into each pipeline iteration. Alternatively, this script can also be used as part of a manual QA process, ensuring that potential bias is identified and addressed before the model reaches production.

How to ensure LLM complies with user privacy and data laws?

When developing LLMs, you need to comply with data protection laws and ethical frameworks and guidelines. Regulations like the GDPR, HIPAA in healthcare, and the AI Act in the EU place significant demands on how personal data is handled, stored, and processed by AI systems. However, adhering to these standards is not as complex as it may seem, especially if you take a strategic approach.

I learned this perspective firsthand during a discussion where Teresa Rodríguez de las Heras, director of the Research Chair UC3M-Microsoft, shared her insights. She remarked: 

The regulatory focus, especially in the draft AI Act, is less on the internal structure of the algorithms (i.e., their code or mathematical models) and more on the practical contexts in which AI is used.

Think about it this way: it is easy to integrate GDPR-compliant services like ChatGPT’s enterprise version or to use AI models in a law-compliant way through platforms such as Azure’s OpenAI offering, as providers take the necessary steps to ensure their platforms are compliant with regulations.

The real challenge lies in how the service is used. While the infrastructure may be compliant, you, as an AI researcher, need to ensure that your LLM’s deployment and data handling practices align with privacy laws. This includes how data is accessed, processed, and stored throughout the model’s lifecycle, as well as thorough documentation of these processes. Clear and detailed documentation is crucial—usually, a technically sound architecture following best practices meets the regulatory requirements, but it has to be documented that it does. By focusing on these aspects, we can shift our understanding of compliance from a purely technical standpoint to a broader, application-based risk perspective, which ultimately affects the overall compliance of your AI system.

You might be wondering, how can I meet these requirements? Here are some security steps you can take to ensure user privacy:

Data anonymization

Protect personal data in your training data by ensuring it is fully anonymized to prevent the leakage of personally identifiable information (PII). Start by:

  • Removing or masking direct identifiers such as names, addresses, emails, job titles, and geographic locations.
  • Using aggregated data instead of raw personal information (e.g., grouping individuals by age ranges or replacing specific locations with broader regions).
  • Applying K-anonymity to generalize or suppress data so each individual cannot be distinguished from at least k-1 others in the dataset.

Once these foundational steps are in place, consider additional measures to limit the risk of re-identification. For practical examples and implementation tips, consider exploring Google’s TensorFlow Privacy repository on GitHub. 

Secure model serving

Ensure that your deployed model is served securely to protect user data during interactions. How?

  • Hosting the model in secure, GDPR-compliant cloud environments, such as Amazon Web Services or Azure.
  • Using encryption protocols like HTTPS and TLS to safeguard data in transit.
  • Implementing access controls to limit who can query the model and monitor interactions.

Privacy penetration tests

Conduct regular privacy penetration tests to identify vulnerabilities in your system. For example:

  • Simulate data extraction attacks to evaluate how well your model resists adversarial attempts to uncover training data. For more information on defending against these threats, check out Defense Strategies in Adversarial Machine Learning.
  • Collaborate with privacy experts to audit your model’s infrastructure and identify potential compliance gaps.

These measures serve as a robust framework for privacy protection without compromising the performance of your LLMs. 

How to integrate transparency, accountability, and explainability?

As LLMs become increasingly integrated into applications and individuals and organizations rely on AI development for their own projects, concerns surrounding the transparency, accountability, and explainability of these systems are growing. 

However, the current market leaves formal interpretability research and solutions mostly in the academic and R&D corners rather than demanding them in everyday products. This makes sense: you don’t need to know where the training data comes from to build an app with ChatGPT, and highly popular tools like GitHub Copilot and Bing Chat thrive without deep interpretability features. That said, certain practical approaches to interpretability (e.g., user-facing explanations for predictions or contextual annotations in outputs) occasionally emerge in industry settings. These glimpses, while rare, provide meaningful transparency and serve specific use cases where interpretability can enhance trust and usability.

Such practical approaches allow users to better understand the results without having to decipher the internal logic. As an AI professional developing LLM-based applications, learning about these strategies—contextual cues, custom filtering, and source references—can differentiate your product. 

Transparency has become a key expectation in the AI industry, as highlighted by initiatives like the EU AI Act and guidelines from organizations such as the Partnership on AI, which emphasize the importance of explainable AI. By integrating them, you can meet these expectations while maintaining feasibility for deployment. Let’s get into it!

What does contextual transparency look like?

Contextual transparency provides meaningful insights into how the model produces outputs, for example, by showing relevant sources, highlighting influential inputs, or offering filtering options. When models display their sources, users can quickly assess their credibility and the accuracy of their results. In cases where the answer is not reliable, these sources are often either fake (links that go nowhere) or redirect to papers or articles unrelated to the topic. You can provide contextual transparency to your LLM by including:

• Disclaimers about outputs: Set expectations by clearly communicating the probabilistic nature of your LLM’s responses and their potential for inaccuracies. OpenAI, for example, includes disclaimers in ChatGPT to guide user understanding. 

OpenAI's ChatGPT disclaimer encouraging users to verify information independently.
OpenAI’s ChatGPT disclaimer encouraging users to verify information independently | Source: Author

While researching for this article, I came across a collection of the best disclaimers from ChatGPT shared by Reddit users. These examples highlight how language models can be prompted to produce disclaimers, though the results don’t always make sense from a human perspective.

• Contextual cues: Contextual cues provide insights about the sources and processes behind the model’s outputs. Features like highlighting citations (as seen in Bing Chat) or referencing snippets of code and links to external materials (as ChatGPT does) help users understand the reasoning behind responses.

• RAG-specific contextualization: In Retrieval-Augmented Generation (RAG) systems, contextualization often involves surfacing top-related documents or tokens that influence the model’s output.

An example of contextual transparency: ChatGPT references the source code in the output.
An example of contextual transparency: ChatGPT references the source code in the output. | Source: Author
An example of contextual transparency: Bing Chat cites the source that influenced its answer.
An example of contextual transparency: Bing Chat cites the source that influenced its answer. | Source

How to navigate data usage risks in AI development?

While regulations often dictate what can be done legally, we also need to consider what should be done to build user trust and ensure fair practices. Deploying ML models implies navigating the line between necessary oversight (e.g., content moderation) and potential overreach. Being AI professionals, we need to approach this challenge responsibly.

Production logs, including user prompts, interactions, and model outputs, offer a wealth of information about the system’s performance and potential misuse. However, they also raise ethical implications about user consent and privacy risks.

Understand your data sources

An important part of building ethically sound AI models lies in verifying that your data comes from sources with clear usage rights. Your data pipeline should flag or exclude content from sources with uncertain copyright status. If you are using scraping tools, start by implementing rules to filter out certain domains or sites that have unclear copyright status. 

Common Crawl is a free, open repository that provides a large dataset of web pages that can be filtered for copyrighted content. While it is a good starting point for identifying general content, I recommend refining these filters with additional checks tailored to your specific topics.

Using publicly accessible data that is copyrighted

The AI industry has faced growing scrutiny over practices like scraping data and using user-provided content without explicit consent. For example, while human users cannot legally reuse or republish copyrighted content from websites or books without explicit permission, many LLM providers use them as training data. The assumption that “publicly accessible” equals “fair use” has led to a growing backlash from creators, publishers, and regulators. Controversial examples include:

Using user data that is not publicly accessible

Some jurisdictions have more robust regulatory frameworks that explicitly regulate how user data can be used to train models. In the EU and the UK, laws like the GDPR have prompted companies to adopt stricter privacy practices. Let’s see some examples:

• Grammarly, for instance, follows a regional approach. It states on its Product Improvement and Training Control page and in the privacy settings that users in the EU and UK automatically have their data excluded from model training:

Since you created your account in the EU or UK, Grammarly will not use your content to train its models or improve its product for other users.

• In 2019, a Bloomberg report revealed that Amazon employees and contractors sometimes review Alexa voice recordings to help improve Alexa’s speech recognition models. While the data review process is intended to enhance product quality, the disclosure raised concerns about user consent, privacy, and the extent to which voice data—often from private homes—could be accessed for AI development. In May 2023, the Federal Trade Commission (FTC) imposed a $25 million fine on Amazon related to children’s privacy, alleging that the company had violated the Children’s Online Privacy Protection Act (COPPA) by retaining children’s voice recordings indefinitely and misrepresenting parents’ ability to delete those recordings.

These examples highlight how regulations differ across jurisdictions. This patchwork of regulations creates a challenging landscape for AI developers, highlighting that what is deemed legal (or even ethical) differs across regions. As a result, some users benefit from stronger protections against such practices than others, depending on their location.

There are some recommendations that may come in handy to navigate different jurisdictions. First, if resources permit, adopt a “highest common denominator” strategy by aligning global practices with the most restrictive data protection requirements (e.g., EU GDPR). Second, keep detailed documentation of each model’s training process—covering data sources, usage procedures, and implemented safeguards—and present this information in an accessible format (e.g., FAQs or transparency reports). This approach demonstrates a clear commitment to transparency and ethical standards.

Best practices for ethical LLM development

Navigating the regulatory landscape requires more than just complying with the local laws. Just as contextual transparency helps users trust the outputs of your LLMs, your broader organizational values, professional standards, or industry best practices form the ethical backbone that ensures this trust extends to the foundation of your system.

By following these practical steps, you can reinforce that commitment to building fair and transparent models:

Implement opt-out mechanisms

Opt-out mechanisms allow users to control whether their data is used to train AI models and other software, giving them some agency over how their data is processed and used. If you plan to store users’ data for training your AI or for any other purpose, implementing an opt-out mechanism is a good practice to give users back control over their personal data. Let’s look at some examples of how this can be done:

  • Social media platforms: Platforms such as Quora, LinkedIn, and Figma have opt-out mechanisms that allow users to request that their data be excluded from certain data mining purposes. However, the specific options and level of transparency can vary widely from platform to platform. Wired has a step-by-step guide on how to stop your data from being used by the most popular platforms to train AI, which I recommend checking out.
  • Opt-out of data scraping: Many websites indicate where or whether they permit automated crawling by providing a “robots.txt” file. While this file signals how a site wishes to be scrapped, it doesn’t technically prevent unauthorized crawlers from harvesting data; compliance ultimately depends on whether the crawler chooses to honor those instructions.
Structure of a 'robots.txt' file
Syntax of a robots-txt file to prevent agents from crawling a website. Each agent is separated in a different line containing its name and the disallow or allow rules attached to it | Source

Keep your documentation updated

Clear and comprehensive documentation can take multiple forms, from end-user guides (explaining the usage and limitations of your LLM) and developer-focused manuals (covering architecture, training procedures, and potential biases) to legal or regulatory documentation for compliance and accountability. 

Model Cards, originally proposed by Margaret Mitchell and Timnit Gebru at Google, offer a structured template for detailing key information about machine learning models: the dataset used, intended use cases, limitations, etc. Hugging Face has implemented a version of Model Cards on its platform, facilitating a standardized way to document Large Language Models (LLMs) and other AI systems. 

By maintaining up-to-date documentation, you help users and stakeholders understand your model’s capabilities and limitations. This plays a crucial role in fostering trust and encouraging responsible use.

For example, OpenAI has publicly documented its red-teaming process, which involves testing models against harmful content to assess their robustness and ethical implications. Documenting such efforts not only promotes transparency but also sets a benchmark for how ethical considerations are addressed in the development process.

Stay ahead of regulations

If your company has a legal team, collaborate with them to ensure compliance with local and international regulations. If not, and you are planning to expand your LLM globally, consider hiring legal advisors to mitigate the legal risks before launching your LLM. 

For example, for applications that are subject to the GDPR, you need to implement and document appropriate technical and organizational measures protecting any personal data you store and process, as outlined in Article 32. These measures often include creating documentation, such as TOM documents, along with terms of service and privacy policies that users must agree to during signup. Adhering to these requirements, particularly in the European context, is essential for building trust and ensuring compliance.

Avoid legal pitfalls that may affect the long-term viability and trustworthiness of your LLMs by anticipating potential regulatory changes. Monitor the legal landscape for AI development in the regions where you currently operate or plan to expand in the future. These are some useful resources:

  • The U.S. National Institute of Standards and Technology (NIST) AI Risk Management Framework is an updated source with recommendations on AI risks and regulatory impacts for individuals and organizations. 

Summing it up: AI ethics done right

Let’s wrap up with a quick recap of all the key takeaways from our discussion:

  • Bias in LLMs is inevitable, but manageable: While algorithmic bias in machine learning models is part of the game, not all biases are negative. Our job is to identify which biases are functional (beneficial to performance) and which ones are harmful (reinforce inequality). Tools like StereoSet and BBQ are useful for pinpointing and mitigating harmful biases.    
  • Protect user privacy from start to finish: Think less about the mathematical structure of your model (that is usually handled by the provider, they will keep it law-compliant) and more about how data is handled in practice during your model’s lifecycle (this is where you are responsible to keep your system law-compliant). Safeguard sensitive information by implementing strong privacy measures like data anonymization, differential privacy, and secure model serving.
  • Transparency is your ally: You don’t have to explain every inner detail of your AI models to be transparent. Instead, focus on providing meaningful insights into how your model produces outputs. Contextual transparency—like source references and disclaimers—builds trust without overwhelming users with technical jargon.
  • Bias mitigation techniques and privacy protection aren’t one-time tasks: They should be continuously integrated throughout your model’s lifecycle. Using tools like Neptune to track and visualize key metrics, including fairness, helps ensure your models stay aligned with ethical standards across iterations and versions.
  • Ethical AI development requires proactive steps: Understand your data sources, implement opt-out mechanisms, keep your documentation up to date, and stay ahead of regulatory changes. Ethical AI isn’t just about compliance—it’s about building trust and accountability with users and stakeholders.

Was the article useful?

Explore more content topics:

Symantec Demonstrates OpenAI’s Operator Agent in PoC Phishing Attack

0

Symantec’s threat hunters have demonstrated how AI agents like OpenAI’s recently launched Operator could be abused for cyberattacks. While AI agents are designed to boost productivity by automating routine tasks, Symantec’s research shows they could also execute complex attack sequences with minimal human input.

This is a big change from older AI models, which could only provide limited help in making harmful content. Symantec’s research came just a day after Tenable Research revealed that the AI chatbot DeepSeek R1 can be misused to generate code for keyloggers and ransomware.

In Symantec’s experiment, the researchers tested Operator’s capabilities by requesting it to:

  • Obtain their email address
  • Create a malicious PowerShell script
  • Send a phishing email containing the script
  • Find a specific employee within their organization

According to Symantec’s blog post, though the Operator initially refused these tasks citing privacy concerns, researchers found that simply stating they had authorization was enough to bypass these ethical safeguards. The AI agent then successfully:

  • Composed and sent a convincing phishing email
  • Determined the email address through pattern analysis
  • Located the target’s information through online searches
  • Created a PowerShell script after researching online resources

Watch as it’s done:

J Stephen Kowski, Field CTO at SlashNext Email Security+, notes that this development requires organizations to strengthen their security measures: “Organizations need to implement robust security controls that assume AI will be used against them, including enhanced email filtering that detects AI-generated content, zero-trust access policies, and continuous security awareness training.”

While current AI agents’ capabilities may seem basic compared to skilled human attackers, their rapid evolution suggests more sophisticated attack scenarios could soon become reality. This might include automated network breaches, infrastructure setup, and prolonged system compromises – all with minimal human intervention.

This research shows that companies need to update their security strategies because AI tools designed to boost productivity can be misused for harmful purposes.


Best Practices to Prevent Theft and Fraud

0

Cybersecurity tips to protect your cryptocurrency from hackers, scams, and fraud. Learn best practices for securing digital assets and staying safe online.

The cryptocurrency market is changing and growing daily, with new coins created weekly. While the broader market is struggling with weak demand and remains at a critical juncture, Cardano’s ADA is among the top-performing altcoins.

The 8th largest cryptocurrency by market cap has made a breakout of the local support of $0.6638 and could test the $0.70 zone soon, according to U.Today. When discussing the bullish outlook for the ADA price prediction, it’s essential to understand that the Securities and Exchange Commission’s potential approval of Grayscale’s Cardano ETF filing can impact liquidity and trading volumes. 

Investors and traders aren’t the only ones interested in cryptocurrency. Hackers are thrilled with the idea of unregulated money, which opens new attack vectors and allows them to disappear, leaving no trace. ADA and other cryptocurrency transactions can’t be reversed, altered, or cancelled.

Once transactions have been written to the blockchain – in other words, confirmed – they become immutable. If threat actors get access to or transfer funds from a victim’s wallet, the money is lost forever. Neither transactions nor accounts are connected to real-world identities, so it’s easy for hackers to remain unidentified when they use cryptocurrency, 

Cyberattacks Are An Ever-Present Threat, And The Crypto World Is No Exception 

No threat facing the world has grown so fast, or in a way as difficult to understand, as the danger from cyberattacks. Investors and traders must understand that the inherent risk is increasing; cybercriminals are getting smarter, and their tactics are becoming more sophisticated. There are multiple types of cyberattacks where malicious actors take advantage of cryptocurrencies, such as: 

  • Ransomware: When ransomware occurs in the crypto space, malware encrypts files containing the victim’s private keys or digital wallet, making them unreadable. The attacker demands a ransom to provide the encryption key. As a rule, hackers demand payment in the form of cryptocurrency. Infection methods include phishing emails, malicious websites, and compromised accounts. 
  • DDoS extorsion: Threat actors blackmail crypto exchanges or blockchain networks by asking them to pay ADA or any other cryptocurrency to avoid their site or service being disrupted by a DDoS attack. The system may struggle to process transactions. Plus, legitimate users can’t connect to network resources.
  • Crypto hijacking: The computing power of a compromised device is used to mine cryptocurrency without the owner’s knowledge (Also called cryptomining). Smartphones, servers, and computers are vulnerable to crypto hijacking. Applications may be able to access data and information from the device or other applications. Unlike malware, control is camouflaged in the background. 

We Suggest These Steps To Secure Your Cryptocurrency 

Transfer Crypto From A Centralized Exchange To A Self-Custody Wallet 

If you wish to own Cardano’s ADA, you can choose from among the many crypto exchanges that provide this service. Of course, you must give priority to the security features. Otherwise, you risk losing your hard-earned savings. Crypto exchanges are some of the most targeted in terms of cyberattacks, and while most attempts by hackers have been successfully mitigated, there have been several reports of damaging attacks. Find a reliable exchange with strong security measures and strong DDoS prevention tools. 

Better yet, withdraw your funds to a secure wallet to stay in control of your assets. Due to the difficulty of effectively implementing safety measures, crypto exchanges can’t fully guarantee protection. Whether they call it withdrawing, sending, or transferring, crypto exchanges let you move your funds to a wallet that is compatible with the asset you’re relocating.

A self-custody wallet puts you in complete control of your cryptocurrency, so you must take full responsibility for the security. Lock your account when not in use to make it harder for others to access your account. 

Keep Safe Backups Of Everything 

No matter what type of wallet you use, having a backup of your data ensures quick recovery and minimizes information loss. Go to your wallet’s settings and select the backup option or the export keys option; encrypt your backups to protect against unauthorized access. If you lose your wallet’s private keys, you’ve permanently lost your cryptocurrency. Keep multiple backups on different devices, such as USB drives and paper, which aren’t prone to failure. This way, you have alternate recovery paths. 

When Dealing With Unsolicited Messages, Be Careful 

One of the most prevalent social engineering techniques, phishing, involves sending deceptive messages that seem to be from legitimate sources. Don’t reply without investigating. Most importantly, never confront the sender of a phishing message yourself because it could result in you being targeted specifically. The sender doesn’t even know if your number is active.

Refrain from opening attachments or clicking on links from unidentified sources even if the sender seems familiar, as you can potentially infect your phone. A phishing link can direct you to a website containing malicious code. You can always use a free site like VirusTotal to scan for malicious files and links before executing/opening them.

Replying to unsolicited messages increases the chances of receiving more spam, so you should block the number. It requires a few adjustments in your settings. Android phones allow for freedom when it comes to customization, so the process will vary from device to device.

Many phones have built-in options to filter out messages from unknown senders or mark texts as spam. Be cautious when sharing your phone number online with unfamiliar organizations because it exposes you to other social engineering-based tactics, such as pretexting, quid pro quo, or vishing. 

In Closing 

Cyberattacks have become increasingly targeted and complex due to sophisticated pieces of malware. Individual users are more vulnerable than ever before since hackers have adapted their strategies to exploit weaknesses in smart contracts, wallets, and decentralized finance (DeFi) platforms. As such, it’s necessary to take personal responsibility and protect yourself. Acquire new skills, expand your knowledge, and, above all, shift your perspective.

Image credits: Free Vector | Realistic cardano coin illustration


Popular Posts

My Favorites

Jeff Bezos’s Blue Origin rocket company beats out spaceflight veteran for...

0
Jeff Bezos's Blue Origin rocket company just scored a major contract....

Reading In The Morning?