首页 ❯

速率限制自己使外部 API 过载

php laravel
2022-08-30 17:53:12

我发现了很多信息和脚本示例，这些示例显示了如何对API的用户进行速率限制，但是我无法找到任何示例，说明在施加这些限制时如何对API的请求进行速率限制。

我总是用代码（如命令）对脚本进行速率限制，但这感觉就像是一种低效的做事方式，特别是当API端点具有相当高的速率限制并且锤击API直到达到限制也是低效的。sleepusleep

例如，Google的API限制根据您使用的API而有所不同，并且可以增加/减少，在这种情况下，硬编码到代码中的固定速率限制似乎只是原始的猜测工作！

我错过了一些非常明显的东西吗？还是这并不像我预期的那么普遍？


答案 1

好吧，对于咯咯笑，我拼凑了一个限制器类，允许您指定每秒，每分钟和每小时的限制。我无法抗拒有一个很好的理由使用循环队列！

如果您有多个进程同时进行消费，无论是否同时进行，您都必须设计一种方法来自行存储和/或共享使用历史记录。

// LIMITER.PHP
class Limiter
{
  private $queue = array();
  private $size;
  private $next;

  private $perSecond;
  private $perMinute;
  private $perHour;

  // Set any constructor parameter to non-zero to allow adherence to the
  // limit represented. The largest value present will be the size of a
  // circular queue used to track usage.
  // -------------------------------------------------------------------
  function __construct($perSecond=0,$perMinute=0,$perHour=0)
  {
    $this->size = max($perSecond,$perMinute,$perHour);
    $this->next = 0;

    $this->perSecond = $perSecond;
    $this->perMinute = $perMinute;
    $this->perHour   = $perHour;

    for($i=0; $i < $this->size; $i++)
      $this->queue[$i] = 0;
  }

  // See if a use would violate any of the limits specified. We return true
  // if a limit has been hit.
  // ----------------------------------------------------------------------
  public function limitHit($verbose=0)
  {    
    $inSecond = 0;
    $inMinute = 0;
    $inHour   = 0;

    $doneSecond = 0;
    $doneMinute = 0;
    $doneHour   = 0;

    $now = microtime(true);

    if ( $verbose )
      echo "Checking if limitHit at $now<br>\n";

    for ($offset=1; $offset <= $this->size; $offset++)
    {
      $spot = $this->next - $offset;
      if ( $spot < 0 )
        $spot = $this->size - $offset + $this->next;

      if ( $verbose )
        echo "... next $this->next size $this->size offset $offset spot $spot utime " . $this->queue[$spot] . "<br>\n";

      // Count and track within second
      // -----------------------------
      if ( $this->perSecond && !$doneSecond && $this->queue[$spot] >= microtime(true) - 1.0 )
        $inSecond++;
      else
        $doneSecond = 1;

      // Count and track within minute
      // -----------------------------
      if ( $this->perMinute && !$doneMinute && $this->queue[$spot] >= microtime(true) - 60.0 )
        $inMinute++;
      else
        $doneMinute = 1;

      // Count and track within hour
      // ---------------------------
      if ( $this->perHour && !$doneHour && $this->queue[$spot] >= microtime(true) - 3600.0 )
        $inHour++;
      else
        $doneHour = 1;

      if ( $doneSecond && $doneMinute && $doneHour )
        break;
    }

    if ( $verbose )
      echo "... inSecond $inSecond inMinute $inMinute inHour $inHour<br>\n";

    if ( $inSecond && $inSecond >= $this->perSecond )
    {
      if ( $verbose )
        echo "... limit perSecond hit<br>\n";
      return TRUE;
    }
    if ( $inMinute && $inMinute >= $this->perMinute )
    {
      if ( $verbose )
        echo "... limit perMinute hit<br>\n";
      return TRUE;
    }
    if ( $inHour   && $inHour   >= $this->perHour   )
    {
      if ( $verbose )
        echo "... limit perHour hit<br>\n";
      return TRUE;
    }

    return FALSE;
  }

  // When an API is called the using program should voluntarily track usage
  // via the use function.
  // ----------------------------------------------------------------------
  public function usage()
  {
    $this->queue[$this->next++] = microtime(true);
    if ( $this->next >= $this->size )
      $this->next = 0;
  }
}

// ##############################
// ### Test the limiter class ###
// ##############################

$psec = 2;
$pmin = 4;
$phr  = 0;

echo "Creating limiter with limits of $psec/sec and $pmin/min and $phr/hr<br><br>\n";
$monitorA = new Limiter($psec,$pmin,$phr);

for ($i=0; $i<15; $i++)
{
  if ( !$monitorA->limitHit(1) )
  {
    echo "<br>\n";
    echo "API call A here (utime " . microtime(true) . ")<br>\n";
    echo "Voluntarily registering usage<br>\n";
    $monitorA->usage();
    usleep(250000);
  }
  else
  {
    echo "<br>\n";
    usleep(500000);
  }
}

为了在操作中演示它，我在限制检查函数中放入了一些“详细模式”语句。下面是一些示例输出。

Creating limiter with limits of 2/sec and 4/min and 0/hr

Checking if limitHit at 1436267440.9957
... next 0 size 4 offset 1 spot 3 utime 0
... inSecond 0 inMinute 0 inHour 0

API call A here (utime 1436267440.9957)
Voluntarily registering usage
Checking if limitHit at 1436267441.2497
... next 1 size 4 offset 1 spot 0 utime 1436267440.9957
... next 1 size 4 offset 2 spot 3 utime 0
... inSecond 1 inMinute 1 inHour 0

API call A here (utime 1436267441.2497)
Voluntarily registering usage
Checking if limitHit at 1436267441.5007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 2 inMinute 2 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267442.0007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 1 inMinute 2 inHour 0

API call A here (utime 1436267442.0007)
Voluntarily registering usage
Checking if limitHit at 1436267442.2507
... next 3 size 4 offset 1 spot 2 utime 1436267442.0007
... next 3 size 4 offset 2 spot 1 utime 1436267441.2497
... next 3 size 4 offset 3 spot 0 utime 1436267440.9957
... next 3 size 4 offset 4 spot 3 utime 0
... inSecond 1 inMinute 3 inHour 0

API call A here (utime 1436267442.2507)
Voluntarily registering usage
Checking if limitHit at 1436267442.5007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267443.0007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267443.5027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit

Checking if limitHit at 1436267444.0027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit

答案 2

好吧，首先 - 你应该只在实际需要的时候调用任何外部API - 提供商会非常感谢你。

我通常有两种方式对自己的API使用“施加”限制 - 如果可能的话，将结果缓存N个时间，通常比API本身的硬限制少得多。但是，这仅适用于非常具体的情况。

第二种是持久/半持久计数器，其中计数器与限制期开始的时间一起存储在某种内存后端。每次调用 API 之前，请检查存储，并查看当前时间减去间隔开始以及您已经发出的请求数是否小于 API 规定的请求数。如果是这样，您可以提出请求 - 如果间隔较大，则可以重置限制，如果您的下一个请求将超过限制，并且您仍然处于上一个间隔，则可以显示一个非常错误。在每个外部请求上，如果超过间隔时间，则更新间隔时间并递增计数器。


推荐
更多 »
标签
更多 »
php pass-by-reference optional-parameters default-value javascript arrays dom function string object serialization tostring visibility syntax jslint use-strict operators equality equality-operator identity-operator datetime timestamp date-arithmetic redirect angularjs loops foreach iteration scope variables
推荐
更多 »